[ 0.000000] Linux version 4.18.0rh8.10-debug (green@maintenance) (gcc version 8.5.0 20210514 (Red Hat 8.5.0-26) (GCC)) #2 SMP Mon Jul 14 01:24:22 EDT 2025 [ 0.000000] Command line: rd.shell root=nbd:192.168.200.253:rocky8.10:ext4:ro:-p,-b4096 ro crashkernel=256M panic=1 nomodeset ipmtu=9000 ip=dhcp rd.neednet=1 init_on_free=off mitigations=off console=ttyS1,115200 audit=0 [ 0.000000] x86/fpu: Supporting XSAVE feature 0x001: 'x87 floating point registers' [ 0.000000] x86/fpu: Supporting XSAVE feature 0x002: 'SSE registers' [ 0.000000] x86/fpu: Supporting XSAVE feature 0x004: 'AVX registers' [ 0.000000] x86/fpu: xstate_offset[2]: 576, xstate_sizes[2]: 256 [ 0.000000] x86/fpu: Enabled xstate features 0x7, context size is 832 bytes, using 'standard' format. [ 0.000000] signal: max sigframe size: 1776 [ 0.000000] BIOS-provided physical RAM map: [ 0.000000] BIOS-e820: [mem 0x0000000000000000-0x000000000009fbff] usable [ 0.000000] BIOS-e820: [mem 0x000000000009fc00-0x000000000009ffff] reserved [ 0.000000] BIOS-e820: [mem 0x00000000000f0000-0x00000000000fffff] reserved [ 0.000000] BIOS-e820: [mem 0x0000000000100000-0x00000000bffcdfff] usable [ 0.000000] BIOS-e820: [mem 0x00000000bffce000-0x00000000bfffffff] reserved [ 0.000000] BIOS-e820: [mem 0x00000000feffc000-0x00000000feffffff] reserved [ 0.000000] BIOS-e820: [mem 0x00000000fffc0000-0x00000000ffffffff] reserved [ 0.000000] BIOS-e820: [mem 0x0000000100000000-0x0000000146dfffff] usable [ 0.000000] NX (Execute Disable) protection: active [ 0.000000] SMBIOS 2.8 present. [ 0.000000] DMI: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.17.0-8.fc42 06/10/2025 [ 0.000000] Hypervisor detected: KVM [ 0.000000] kvm-clock: Using msrs 4b564d01 and 4b564d00 [ 0.000000] kvm-clock: using sched offset of 496449843 cycles [ 0.000000] clocksource: kvm-clock: mask: 0xffffffffffffffff max_cycles: 0x1cd42e4dffb, max_idle_ns: 881590591483 ns [ 0.000000] tsc: Detected 2399.998 MHz processor [ 0.000000] last_pfn = 0x146e00 max_arch_pfn = 0x400000000 [ 0.000000] x86/PAT: Configuration [0-7]: WB WC UC- UC WB WP UC- WT [ 0.000000] last_pfn = 0xbffce max_arch_pfn = 0x400000000 [ 0.000000] found SMP MP-table at [mem 0x000f54b0-0x000f54bf] [ 0.000000] RAMDISK: [mem 0xbcc54000-0xbffbffff] [ 0.000000] ACPI: Early table checksum verification disabled [ 0.000000] ACPI: RSDP 0x00000000000F52D0 000014 (v00 BOCHS ) [ 0.000000] ACPI: RSDT 0x00000000BFFE2439 000034 (v01 BOCHS BXPC 00000001 BXPC 00000001) [ 0.000000] ACPI: FACP 0x00000000BFFE22D5 000074 (v01 BOCHS BXPC 00000001 BXPC 00000001) [ 0.000000] ACPI: DSDT 0x00000000BFFE0040 002295 (v01 BOCHS BXPC 00000001 BXPC 00000001) [ 0.000000] ACPI: FACS 0x00000000BFFE0000 000040 [ 0.000000] ACPI: APIC 0x00000000BFFE2349 000090 (v03 BOCHS BXPC 00000001 BXPC 00000001) [ 0.000000] ACPI: HPET 0x00000000BFFE23D9 000038 (v01 BOCHS BXPC 00000001 BXPC 00000001) [ 0.000000] ACPI: WAET 0x00000000BFFE2411 000028 (v01 BOCHS BXPC 00000001 BXPC 00000001) [ 0.000000] ACPI: Reserving FACP table memory at [mem 0xbffe22d5-0xbffe2348] [ 0.000000] ACPI: Reserving DSDT table memory at [mem 0xbffe0040-0xbffe22d4] [ 0.000000] ACPI: Reserving FACS table memory at [mem 0xbffe0000-0xbffe003f] [ 0.000000] ACPI: Reserving APIC table memory at [mem 0xbffe2349-0xbffe23d8] [ 0.000000] ACPI: Reserving HPET table memory at [mem 0xbffe23d9-0xbffe2410] [ 0.000000] ACPI: Reserving WAET table memory at [mem 0xbffe2411-0xbffe2438] [ 0.000000] No NUMA configuration found [ 0.000000] Faking a node at [mem 0x0000000000000000-0x0000000146dfffff] [ 0.000000] NODE_DATA(0) allocated [mem 0x1465a3000-0x1465cdfff] [ 0.000000] Reserving 256MB of memory at 2752MB for crashkernel (System RAM: 4205MB) [ 0.000000] Zone ranges: [ 0.000000] DMA [mem 0x0000000000001000-0x0000000000ffffff] [ 0.000000] DMA32 [mem 0x0000000001000000-0x00000000ffffffff] [ 0.000000] Normal [mem 0x0000000100000000-0x0000000146dfffff] [ 0.000000] Device empty [ 0.000000] Movable zone start for each node [ 0.000000] Early memory node ranges [ 0.000000] node 0: [mem 0x0000000000001000-0x000000000009efff] [ 0.000000] node 0: [mem 0x0000000000100000-0x00000000bffcdfff] [ 0.000000] node 0: [mem 0x0000000100000000-0x0000000146dfffff] [ 0.000000] Zeroed struct page in unavailable ranges: 4756 pages [ 0.000000] Initmem setup node 0 [mem 0x0000000000001000-0x0000000146dfffff] [ 0.000000] ACPI: PM-Timer IO Port: 0x608 [ 0.000000] ACPI: LAPIC_NMI (acpi_id[0xff] dfl dfl lint[0x1]) [ 0.000000] IOAPIC[0]: apic_id 0, version 17, address 0xfec00000, GSI 0-23 [ 0.000000] ACPI: INT_SRC_OVR (bus 0 bus_irq 0 global_irq 2 dfl dfl) [ 0.000000] ACPI: INT_SRC_OVR (bus 0 bus_irq 5 global_irq 5 high level) [ 0.000000] ACPI: INT_SRC_OVR (bus 0 bus_irq 9 global_irq 9 high level) [ 0.000000] ACPI: INT_SRC_OVR (bus 0 bus_irq 10 global_irq 10 high level) [ 0.000000] ACPI: INT_SRC_OVR (bus 0 bus_irq 11 global_irq 11 high level) [ 0.000000] Using ACPI (MADT) for SMP configuration information [ 0.000000] ACPI: HPET id: 0x8086a201 base: 0xfed00000 [ 0.000000] TSC deadline timer available [ 0.000000] smpboot: Allowing 4 CPUs, 0 hotplug CPUs [ 0.000000] kvm-guest: KVM setup pv remote TLB flush [ 0.000000] kvm-guest: setup PV sched yield [ 0.000000] PM: Registered nosave memory: [mem 0x00000000-0x00000fff] [ 0.000000] PM: Registered nosave memory: [mem 0x0009f000-0x0009ffff] [ 0.000000] PM: Registered nosave memory: [mem 0x000a0000-0x000effff] [ 0.000000] PM: Registered nosave memory: [mem 0x000f0000-0x000fffff] [ 0.000000] PM: Registered nosave memory: [mem 0xbffce000-0xbfffffff] [ 0.000000] PM: Registered nosave memory: [mem 0xc0000000-0xfeffbfff] [ 0.000000] PM: Registered nosave memory: [mem 0xfeffc000-0xfeffffff] [ 0.000000] PM: Registered nosave memory: [mem 0xff000000-0xfffbffff] [ 0.000000] PM: Registered nosave memory: [mem 0xfffc0000-0xffffffff] [ 0.000000] [mem 0xc0000000-0xfeffbfff] available for PCI devices [ 0.000000] Booting paravirtualized kernel on KVM [ 0.000000] clocksource: refined-jiffies: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 1910969940391419 ns [ 0.000000] setup_percpu: NR_CPUS:8192 nr_cpumask_bits:4 nr_cpu_ids:4 nr_node_ids:1 [ 0.000000] percpu: Embedded 63 pages/cpu s221184 r8192 d28672 u524288 [ 0.000000] kvm-guest: PV spinlocks enabled [ 0.000000] PV qspinlock hash table entries: 256 (order: 0, 4096 bytes, linear) [ 0.000000] Built 1 zonelists, mobility grouping on. Total pages: 1059606 [ 0.000000] Policy zone: Normal [ 0.000000] Kernel command line: rd.shell root=nbd:192.168.200.253:rocky8.10:ext4:ro:-p,-b4096 ro crashkernel=256M panic=1 nomodeset ipmtu=9000 ip=dhcp rd.neednet=1 init_on_free=off mitigations=off console=ttyS1,115200 audit=0 [ 0.000000] Specific versions of hardware are certified with Red Hat Enterprise Linux 8. Please see the list of hardware certified with Red Hat Enterprise Linux 8 at https://catalog.redhat.com. [ 0.000000] audit: disabled (until reboot) [ 0.000000] software IO TLB: area num 4. [ 0.000000] Memory: 2829652K/4306352K available (18435K kernel code, 11221K rwdata, 7248K rodata, 2908K init, 18040K bss, 524580K reserved, 0K cma-reserved) [ 0.000000] SLUB: HWalign=64, Order=0-3, MinObjects=0, CPUs=4, Nodes=1 [ 0.000000] kmemleak: Kernel memory leak detector disabled [ 0.000000] ftrace: allocating 41240 entries in 162 pages [ 0.000000] ftrace: allocated 162 pages with 3 groups [ 0.000000] rcu: Hierarchical RCU implementation. [ 0.000000] rcu: RCU event tracing is enabled. [ 0.000000] rcu: RCU restricting CPUs from NR_CPUS=8192 to nr_cpu_ids=4. [ 0.000000] rcu: RCU callback double-/use-after-free debug enabled. [ 0.000000] Rude variant of Tasks RCU enabled. [ 0.000000] Tracing variant of Tasks RCU enabled. [ 0.000000] rcu: RCU calculated value of scheduler-enlistment delay is 100 jiffies. [ 0.000000] rcu: Adjusting geometry for rcu_fanout_leaf=16, nr_cpu_ids=4 [ 0.000000] NR_IRQS: 524544, nr_irqs: 456, preallocated irqs: 16 [ 0.000000] random: get_random_bytes called from start_kernel+0x622/0x9a8 with crng_init=0 [ 0.001000] Console: colour *CGA 80x25 [ 0.001000] printk: console [ttyS1] enabled [ 0.001000] ACPI: Core revision 20220331 [ 0.001000] clocksource: hpet: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 19112604467 ns [ 0.001014] APIC: Switch to symmetric I/O mode setup [ 0.002324] x2apic enabled [ 0.003007] Switched APIC routing to physical x2apic. [ 0.004016] kvm-guest: setup PV IPIs [ 0.007508] ..TIMER: vector=0x30 apic1=0 pin1=2 apic2=-1 pin2=-1 [ 0.008000] clocksource: tsc-early: mask: 0xffffffffffffffff max_cycles: 0x229835b7123, max_idle_ns: 440795242976 ns [ 0.008030] Calibrating delay loop (skipped) preset value.. 4799.99 BogoMIPS (lpj=2399998) [ 0.009020] pid_max: default: 32768 minimum: 301 [ 0.011174] LSM: Security Framework initializing [ 0.013048] Yama: becoming mindful. [ 0.014060] SELinux: Initializing. [ 0.015078] *** VALIDATE selinux *** [ 0.023766] Dentry cache hash table entries: 1048576 (order: 11, 8388608 bytes, vmalloc) [ 0.028493] Inode-cache hash table entries: 524288 (order: 10, 4194304 bytes, vmalloc) [ 0.029185] Mount-cache hash table entries: 16384 (order: 5, 131072 bytes, vmalloc) [ 0.030128] Mountpoint-cache hash table entries: 16384 (order: 5, 131072 bytes, vmalloc) [ 0.031171] *** VALIDATE tmpfs *** [ 0.032525] *** VALIDATE proc *** [ 0.033308] *** VALIDATE cgroup *** [ 0.034013] *** VALIDATE cgroup2 *** [ 0.035317] x86/cpu: User Mode Instruction Prevention (UMIP) activated [ 0.036192] Last level iTLB entries: 4KB 0, 2MB 0, 4MB 0 [ 0.037016] Last level dTLB entries: 4KB 0, 2MB 0, 4MB 0, 1GB 0 [ 0.038044] Spectre V2 : User space: Vulnerable [ 0.039016] Speculative Store Bypass: Vulnerable [ 0.042263] debug: unmapping init [mem 0xffffffff9d659000-0xffffffff9d660fff] [ 0.044340] smpboot: CPU0: Intel(R) Xeon(R) CPU E5-2695 v2 @ 2.40GHz (family: 0x6, model: 0x3e, stepping: 0x4) [ 0.045829] Performance Events: IvyBridge events, full-width counters, Intel PMU driver. [ 0.046037] ... version: 2 [ 0.047021] ... bit width: 48 [ 0.048022] ... generic registers: 4 [ 0.049022] ... value mask: 0000ffffffffffff [ 0.050025] ... max period: 00007fffffffffff [ 0.051020] ... fixed-purpose events: 3 [ 0.052018] ... event mask: 000000070000000f [ 0.053377] rcu: Hierarchical SRCU implementation. [ 0.055836] smp: Bringing up secondary CPUs ... [ 0.056703] x86: Booting SMP configuration: [ 0.057051] .... node #0, CPUs: #1 #2 #3 [ 0.060594] smp: Brought up 1 node, 4 CPUs [ 0.062018] smpboot: Max logical packages: 1 [ 0.063030] smpboot: Total of 4 processors activated (19199.98 BogoMIPS) [ 0.252966] node 0 deferred pages initialised in 188ms [ 0.256185] devtmpfs: initialized [ 0.258339] x86/mm: Memory block size: 128MB [ 0.261382] gcov: version magic: 0x41383552 [ 0.265310] clocksource: jiffies: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 1911260446275000 ns [ 0.266088] futex hash table entries: 1024 (order: 4, 65536 bytes, vmalloc) [ 0.267292] pinctrl core: initialized pinctrl subsystem [ 0.268393] [ 0.269011] ************************************************************* [ 0.271017] ** NOTICE NOTICE NOTICE NOTICE NOTICE NOTICE NOTICE ** [ 0.274018] ** ** [ 0.276015] ** IOMMU DebugFS SUPPORT HAS BEEN ENABLED IN THIS KERNEL ** [ 0.279019] ** ** [ 0.281013] ** This means that this kernel is built to expose internal ** [ 0.283016] ** IOMMU data structures, which may compromise security on ** [ 0.286018] ** your system. ** [ 0.288015] ** ** [ 0.291018] ** If you see this message and you are not debugging the ** [ 0.293014] ** kernel, report this immediately to your vendor! ** [ 0.295015] ** ** [ 0.298018] ** NOTICE NOTICE NOTICE NOTICE NOTICE NOTICE NOTICE ** [ 0.300029] ************************************************************* [ 0.304148] NET: Registered protocol family 16 [ 0.306572] DMA: preallocated 512 KiB GFP_KERNEL pool for atomic allocations [ 0.308075] DMA: preallocated 512 KiB GFP_KERNEL|GFP_DMA pool for atomic allocations [ 0.310067] DMA: preallocated 512 KiB GFP_KERNEL|GFP_DMA32 pool for atomic allocations [ 0.314095] cpuidle: using governor menu [ 0.316886] acpiphp: ACPI Hot Plug PCI Controller Driver version: 0.5 [ 0.319512] PCI: Using configuration type 1 for base access [ 0.321101] core: PMU erratum BJ122, BV98, HSD29 worked around, HT is on [ 0.330135] HugeTLB registered 1.00 GiB page size, pre-allocated 0 pages [ 0.332028] HugeTLB registered 2.00 MiB page size, pre-allocated 0 pages [ 0.336048] cryptd: max_cpu_qlen set to 1000 [ 0.339323] ACPI: Added _OSI(Module Device) [ 0.340013] ACPI: Added _OSI(Processor Device) [ 0.342016] ACPI: Added _OSI(3.0 _SCP Extensions) [ 0.344017] ACPI: Added _OSI(Processor Aggregator Device) [ 0.349141] ACPI: 1 ACPI AML tables successfully acquired and loaded [ 0.355556] ACPI: Interpreter enabled [ 0.357080] ACPI: PM: (supports S0 S3 S4 S5) [ 0.359016] ACPI: Using IOAPIC for interrupt routing [ 0.360120] PCI: Using host bridge windows from ACPI; if necessary, use "pci=nocrs" and report a bug [ 0.362644] ACPI: Enabled 2 GPEs in block 00 to 0F [ 0.373161] ACPI: PCI Root Bridge [PCI0] (domain 0000 [bus 00-ff]) [ 0.375053] acpi PNP0A03:00: _OSC: OS supports [ASPM ClockPM Segments MSI HPX-Type3] [ 0.378022] acpi PNP0A03:00: _OSC: not requesting OS control; OS requires [ExtendedConfig ASPM ClockPM MSI] [ 0.381113] acpi PNP0A03:00: fail to add MMCONFIG information, can't access extended PCI configuration space under this bridge. [ 0.386504] acpiphp: Slot [2] registered [ 0.388146] acpiphp: Slot [5] registered [ 0.390148] acpiphp: Slot [6] registered [ 0.391171] acpiphp: Slot [7] registered [ 0.393134] acpiphp: Slot [8] registered [ 0.394139] acpiphp: Slot [9] registered [ 0.396118] acpiphp: Slot [10] registered [ 0.397131] acpiphp: Slot [3] registered [ 0.399123] acpiphp: Slot [4] registered [ 0.400197] acpiphp: Slot [11] registered [ 0.402119] acpiphp: Slot [12] registered [ 0.404113] acpiphp: Slot [13] registered [ 0.405116] acpiphp: Slot [14] registered [ 0.407108] acpiphp: Slot [15] registered [ 0.408120] acpiphp: Slot [16] registered [ 0.410817] acpiphp: Slot [17] registered [ 0.412118] acpiphp: Slot [18] registered [ 0.414127] acpiphp: Slot [19] registered [ 0.415126] acpiphp: Slot [20] registered [ 0.417111] acpiphp: Slot [21] registered [ 0.418099] acpiphp: Slot [22] registered [ 0.419146] acpiphp: Slot [23] registered [ 0.421152] acpiphp: Slot [24] registered [ 0.422204] acpiphp: Slot [25] registered [ 0.424136] acpiphp: Slot [26] registered [ 0.425197] acpiphp: Slot [27] registered [ 0.427125] acpiphp: Slot [28] registered [ 0.429131] acpiphp: Slot [29] registered [ 0.430161] acpiphp: Slot [30] registered [ 0.432139] acpiphp: Slot [31] registered [ 0.433129] PCI host bridge to bus 0000:00 [ 0.434024] pci_bus 0000:00: root bus resource [io 0x0000-0x0cf7 window] [ 0.436023] pci_bus 0000:00: root bus resource [io 0x0d00-0xffff window] [ 0.438033] pci_bus 0000:00: root bus resource [mem 0x000a0000-0x000bffff window] [ 0.441028] pci_bus 0000:00: root bus resource [mem 0xc0000000-0xfebfffff window] [ 0.443038] pci_bus 0000:00: root bus resource [mem 0xe0000000000-0xe007fffffff window] [ 0.446045] pci_bus 0000:00: root bus resource [bus 00-ff] [ 0.448284] pci 0000:00:00.0: [8086:1237] type 00 class 0x060000 [ 0.451304] pci 0000:00:01.0: [8086:7000] type 00 class 0x060100 [ 0.455096] pci 0000:00:01.1: [8086:7010] type 00 class 0x010180 [ 0.464017] pci 0000:00:01.1: reg 0x20: [io 0xc320-0xc32f] [ 0.469136] pci 0000:00:01.1: legacy IDE quirk: reg 0x10: [io 0x01f0-0x01f7] [ 0.473024] pci 0000:00:01.1: legacy IDE quirk: reg 0x14: [io 0x03f6] [ 0.475025] pci 0000:00:01.1: legacy IDE quirk: reg 0x18: [io 0x0170-0x0177] [ 0.477022] pci 0000:00:01.1: legacy IDE quirk: reg 0x1c: [io 0x0376] [ 0.480262] pci 0000:00:01.3: [8086:7113] type 00 class 0x068000 [ 0.482887] pci 0000:00:01.3: quirk: [io 0x0600-0x063f] claimed by PIIX4 ACPI [ 0.486285] pci 0000:00:01.3: quirk: [io 0x0700-0x070f] claimed by PIIX4 SMB [ 0.489918] pci 0000:00:02.0: [1af4:1000] type 00 class 0x020000 [ 0.495019] pci 0000:00:02.0: reg 0x10: [io 0xc300-0xc31f] [ 0.509018] pci 0000:00:02.0: reg 0x20: [mem 0xe0000000000-0xe0000003fff 64bit pref] [ 0.515955] pci 0000:00:02.0: reg 0x30: [mem 0xfeb80000-0xfebbffff pref] [ 0.522407] pci 0000:00:05.0: [1af4:1001] type 00 class 0x010000 [ 0.529019] pci 0000:00:05.0: reg 0x10: [io 0xc000-0xc07f] [ 0.538022] pci 0000:00:05.0: reg 0x14: [mem 0xfebc0000-0xfebc0fff] [ 0.556022] pci 0000:00:05.0: reg 0x20: [mem 0xe0000004000-0xe0000007fff 64bit pref] [ 0.566905] pci 0000:00:06.0: [1af4:1001] type 00 class 0x010000 [ 0.573024] pci 0000:00:06.0: reg 0x10: [io 0xc080-0xc0ff] [ 0.581026] pci 0000:00:06.0: reg 0x14: [mem 0xfebc1000-0xfebc1fff] [ 0.597025] pci 0000:00:06.0: reg 0x20: [mem 0xe0000008000-0xe000000bfff 64bit pref] [ 0.609020] pci 0000:00:07.0: [1af4:1001] type 00 class 0x010000 [ 0.619020] pci 0000:00:07.0: reg 0x10: [io 0xc100-0xc17f] [ 0.627024] pci 0000:00:07.0: reg 0x14: [mem 0xfebc2000-0xfebc2fff] [ 0.647026] pci 0000:00:07.0: reg 0x20: [mem 0xe000000c000-0xe000000ffff 64bit pref] [ 0.660033] pci 0000:00:08.0: [1af4:1001] type 00 class 0x010000 [ 0.669019] pci 0000:00:08.0: reg 0x10: [io 0xc180-0xc1ff] [ 0.675019] pci 0000:00:08.0: reg 0x14: [mem 0xfebc3000-0xfebc3fff] [ 0.689017] pci 0000:00:08.0: reg 0x20: [mem 0xe0000010000-0xe0000013fff 64bit pref] [ 0.698908] pci 0000:00:09.0: [1af4:1001] type 00 class 0x010000 [ 0.707020] pci 0000:00:09.0: reg 0x10: [io 0xc200-0xc27f] [ 0.715024] pci 0000:00:09.0: reg 0x14: [mem 0xfebc4000-0xfebc4fff] [ 0.740025] pci 0000:00:09.0: reg 0x20: [mem 0xe0000014000-0xe0000017fff 64bit pref] [ 0.758056] pci 0000:00:0a.0: [1af4:1001] type 00 class 0x010000 [ 0.765018] pci 0000:00:0a.0: reg 0x10: [io 0xc280-0xc2ff] [ 0.771020] pci 0000:00:0a.0: reg 0x14: [mem 0xfebc5000-0xfebc5fff] [ 0.786032] pci 0000:00:0a.0: reg 0x20: [mem 0xe0000018000-0xe000001bfff 64bit pref] [ 0.797420] ACPI: PCI: Interrupt link LNKA configured for IRQ 10 [ 0.800422] ACPI: PCI: Interrupt link LNKB configured for IRQ 10 [ 0.803361] ACPI: PCI: Interrupt link LNKC configured for IRQ 11 [ 0.805376] ACPI: PCI: Interrupt link LNKD configured for IRQ 11 [ 0.807263] ACPI: PCI: Interrupt link LNKS configured for IRQ 9 [ 0.812157] iommu: Default domain type: Passthrough [ 0.813438] SCSI subsystem initialized [ 0.815147] ACPI: bus type USB registered [ 0.816138] usbcore: registered new interface driver usbfs [ 0.818083] usbcore: registered new interface driver hub [ 0.820101] usbcore: registered new device driver usb [ 0.822215] pps_core: LinuxPPS API ver. 1 registered [ 0.823010] pps_core: Software ver. 5.3.6 - Copyright 2005-2007 Rodolfo Giometti [ 0.826072] PTP clock support registered [ 0.828171] EDAC MC: Ver: 3.0.0 [ 0.830147] PCI: Using ACPI for IRQ routing [ 0.831956] NetLabel: Initializing [ 0.833010] NetLabel: domain hash size = 128 [ 0.833972] NetLabel: protocols = UNLABELED CIPSOv4 CALIPSO [ 0.835097] NetLabel: unlabeled traffic allowed by default [ 0.837147] vgaarb: loaded [ 0.838316] hpet0: at MMIO 0xfed00000, IRQs 2, 8, 0 [ 0.839014] hpet0: 3 comparators, 64-bit 100.000000 MHz counter [ 0.845279] clocksource: Switched to clocksource kvm-clock [ 0.953267] VFS: Disk quotas dquot_6.6.0 [ 0.954430] VFS: Dquot-cache hash table entries: 512 (order 0, 4096 bytes) [ 0.956078] *** VALIDATE ramfs *** [ 0.956856] *** VALIDATE hugetlbfs *** [ 0.957979] pnp: PnP ACPI init [ 0.959821] pnp: PnP ACPI: found 6 devices [ 0.978982] clocksource: acpi_pm: mask: 0xffffff max_cycles: 0xffffff, max_idle_ns: 2085701024 ns [ 0.981953] pci_bus 0000:00: resource 4 [io 0x0000-0x0cf7 window] [ 0.984050] pci_bus 0000:00: resource 5 [io 0x0d00-0xffff window] [ 0.986132] pci_bus 0000:00: resource 6 [mem 0x000a0000-0x000bffff window] [ 0.988526] pci_bus 0000:00: resource 7 [mem 0xc0000000-0xfebfffff window] [ 0.990695] pci_bus 0000:00: resource 8 [mem 0xe0000000000-0xe007fffffff window] [ 0.993651] NET: Registered protocol family 2 [ 0.996155] IP idents hash table entries: 131072 (order: 8, 1048576 bytes, vmalloc) [ 1.000857] tcp_listen_portaddr_hash hash table entries: 4096 (order: 5, 163840 bytes, vmalloc) [ 1.003459] TCP established hash table entries: 65536 (order: 7, 524288 bytes, vmalloc) [ 1.007758] TCP bind hash table entries: 65536 (order: 9, 2097152 bytes, vmalloc) [ 1.011160] TCP: Hash tables configured (established 65536 bind 65536) [ 1.014114] MPTCP token hash table entries: 8192 (order: 6, 393216 bytes, vmalloc) [ 1.017131] UDP hash table entries: 4096 (order: 6, 393216 bytes, vmalloc) [ 1.020109] UDP-Lite hash table entries: 4096 (order: 6, 393216 bytes, vmalloc) [ 1.023074] NET: Registered protocol family 1 [ 1.026377] RPC: Registered named UNIX socket transport module. [ 1.028388] RPC: Registered udp transport module. [ 1.030100] RPC: Registered tcp transport module. [ 1.031857] RPC: Registered tcp NFSv4.1 backchannel transport module. [ 1.034152] NET: Registered protocol family 44 [ 1.035647] pci 0000:00:00.0: Limiting direct PCI/PCI transfers [ 1.037703] pci 0000:00:01.0: PIIX3: Enabling Passive Release [ 1.039846] pci 0000:00:01.0: Activating ISA DMA hang workarounds [ 1.042292] PCI: CLS 0 bytes, default 64 [ 1.043938] Unpacking initramfs... [ 2.491632] debug: unmapping init [mem 0xffff9c13bcc54000-0xffff9c13bffbffff] [ 2.495864] PCI-DMA: Using software bounce buffering for IO (SWIOTLB) [ 2.498552] software IO TLB: mapped [mem 0x00000000a0c00000-0x00000000a4c00000] (64MB) [ 2.501941] clocksource: tsc: mask: 0xffffffffffffffff max_cycles: 0x229835b7123, max_idle_ns: 440795242976 ns [ 3.062296] Initialise system trusted keyrings [ 3.065473] Key type blacklist registered [ 3.067541] workingset: timestamp_bits=36 max_order=20 bucket_order=0 [ 3.077301] zbud: loaded [ 3.080635] *** VALIDATE nfs *** [ 3.082119] *** VALIDATE nfs4 *** [ 3.083839] pstore: using deflate compression [ 3.087782] Platform Keyring initialized [ 3.219860] NET: Registered protocol family 38 [ 3.221853] Key type asymmetric registered [ 3.222985] Asymmetric key parser 'x509' registered [ 3.224696] Block layer SCSI generic (bsg) driver version 0.4 loaded (major 247) [ 3.227870] io scheduler mq-deadline registered [ 3.229890] io scheduler kyber registered [ 3.232755] io scheduler bfq registered [ 3.235296] atomic64_test: passed for x86-64 platform with CX8 and with SSE [ 3.237272] shpchp: Standard Hot Plug PCI Controller Driver version: 0.4 [ 3.239303] input: Power Button as /devices/LNXSYSTM:00/LNXPWRBN:00/input/input0 [ 3.241912] ACPI: Power Button [PWRF] [ 3.249243] ACPI: \_SB_.LNKB: Enabled at IRQ 10 [ 3.256632] ACPI: \_SB_.LNKA: Enabled at IRQ 11 [ 3.277166] ACPI: \_SB_.LNKC: Enabled at IRQ 11 [ 3.284230] ACPI: \_SB_.LNKD: Enabled at IRQ 10 [ 3.313388] Serial: 8250/16550 driver, 4 ports, IRQ sharing enabled [ 3.340019] 00:03: ttyS1 at I/O 0x2f8 (irq = 3, base_baud = 115200) is a 16550A [ 3.367071] 00:04: ttyS0 at I/O 0x3f8 (irq = 4, base_baud = 115200) is a 16550A [ 3.372555] Non-volatile memory driver v1.3 [ 3.374065] Linux agpgart interface v0.103 [ 3.410041] virtio_blk virtio1: [vda] 139368 512-byte logical blocks (71.4 MB/68.1 MiB) [ 3.413306] vda: detected capacity change from 0 to 71356416 [ 3.427954] virtio_blk virtio2: [vdb] 2097152 512-byte logical blocks (1.07 GB/1.00 GiB) [ 3.430609] vdb: detected capacity change from 0 to 1073741824 [ 3.444331] virtio_blk virtio3: [vdc] 5120000 512-byte logical blocks (2.62 GB/2.44 GiB) [ 3.446884] vdc: detected capacity change from 0 to 2621440000 [ 3.462546] virtio_blk virtio4: [vdd] 5120000 512-byte logical blocks (2.62 GB/2.44 GiB) [ 3.465538] vdd: detected capacity change from 0 to 2621440000 [ 3.482399] virtio_blk virtio5: [vde] 8388608 512-byte logical blocks (4.29 GB/4.00 GiB) [ 3.485459] vde: detected capacity change from 0 to 4294967296 [ 3.501531] virtio_blk virtio6: [vdf] 8388608 512-byte logical blocks (4.29 GB/4.00 GiB) [ 3.504551] vdf: detected capacity change from 0 to 4294967296 [ 3.512610] libphy: Fixed MDIO Bus: probed [ 3.518514] usbcore: registered new interface driver usbserial_generic [ 3.521236] usbserial: USB Serial support registered for generic [ 3.523909] i8042: PNP: PS/2 Controller [PNP0303:KBD,PNP0f13:MOU] at 0x60,0x64 irq 1,12 [ 3.527787] serio: i8042 KBD port at 0x60,0x64 irq 1 [ 3.530138] serio: i8042 AUX port at 0x60,0x64 irq 12 [ 3.532313] mousedev: PS/2 mouse device common for all mice [ 3.535131] rtc_cmos 00:05: RTC can wake from S4 [ 3.538611] input: AT Translated Set 2 keyboard as /devices/platform/i8042/serio0/input/input1 [ 3.543975] rtc_cmos 00:05: registered as rtc0 [ 3.545661] rtc_cmos 00:05: alarms up to one day, y3k, 242 bytes nvram, hpet irqs [ 3.547753] intel_pstate: CPU model not supported [ 3.550302] input: VirtualPS/2 VMware VMMouse as /devices/platform/i8042/serio1/input/input4 [ 3.554815] hid: raw HID events driver (C) Jiri Kosina [ 3.556519] usbcore: registered new interface driver usbhid [ 3.556571] input: VirtualPS/2 VMware VMMouse as /devices/platform/i8042/serio1/input/input3 [ 3.558282] usbhid: USB HID core driver [ 3.563135] drop_monitor: Initializing network drop monitor service [ 3.565477] Initializing XFRM netlink socket [ 3.567350] NET: Registered protocol family 10 [ 3.570684] Segment Routing with IPv6 [ 3.573916] NET: Registered protocol family 17 [ 3.576229] mpls_gso: MPLS GSO support [ 3.581595] RAS: Correctable Errors collector initialized. [ 3.583872] AVX version of gcm_enc/dec engaged. [ 3.585557] AES CTR mode by8 optimization enabled [ 3.684494] sched_clock: Marking stable (3684468615, 0)->(4640124538, -955655923) [ 3.688502] registered taskstats version 1 [ 3.690461] Loading compiled-in X.509 certificates [ 3.692456] zswap: loaded using pool lzo/zbud [ 3.719967] Key type big_key registered [ 3.735363] Key type encrypted registered [ 3.737145] ima: No TPM chip found, activating TPM-bypass! [ 3.739611] ima: Allocated hash algorithm: sha1 [ 3.741592] ima: No architecture policies found [ 3.743676] evm: Initialising EVM extended attributes: [ 3.745427] evm: security.selinux [ 3.746751] evm: security.ima [ 3.747871] evm: security.capability [ 3.749315] evm: HMAC attrs: 0x1 [ 3.752279] rtc_cmos 00:05: setting system clock to 2026-02-10 00:52:39 UTC (1770684759) [ 3.758377] debug: unmapping init [mem 0xffffffff9e603000-0xffffffff9e7fffff] [ 3.761362] debug: unmapping init [mem 0xffffffff9d382000-0xffffffff9d658fff] [ 3.766082] Write protecting the kernel read-only data: 28672k [ 3.769203] debug: unmapping init [mem 0xffffffff9ba03000-0xffffffff9bbfffff] [ 3.771309] debug: unmapping init [mem 0xffffffff9c314000-0xffffffff9c3fffff] [ 3.812183] systemd[1]: systemd 239 (239-82.el8_10.5) running in system mode. (+PAM +AUDIT +SELINUX +IMA -APPARMOR +SMACK +SYSVINIT +UTMP +LIBCRYPTSETUP +GCRYPT +GNUTLS +ACL +XZ +LZ4 +SECCOMP +BLKID +ELFUTILS +KMOD +IDN2 -IDN +PCRE2 default-hierarchy=legacy) [ 3.819835] systemd[1]: Detected virtualization kvm. [ 3.821799] systemd[1]: Detected architecture x86-64. [ 3.823444] systemd[1]: Running in initial RAM disk. Welcome to Rocky Linux 8.10 (Green Obsidian) dracut-049-233.git20240115.el8 (Initramfs)! [ 3.844877] systemd[1]: No hostname configured. [ 3.846123] systemd[1]: Set hostname to . [ 3.847831] random: systemd: uninitialized urandom read (16 bytes read) [ 3.849914] systemd[1]: Initializing machine ID from random generator. [ 3.889202] random: ln: uninitialized urandom read (6 bytes read) [ 4.000249] random: systemd: uninitialized urandom read (16 bytes read) [ 4.002833] systemd[1]: Reached target Initrd Root Device. [ OK ] Reached target Initrd Root Device. [ 4.007050] systemd[1]: Listening on udev Kernel Socket. [ OK ] Listening on udev Kernel Socket. [ 4.011424] systemd[1]: Reached target Slices. [ OK ] Reached target Slices. [ OK ] Reached target Timers. [ OK ] Listening on Journal Socket (/dev/log). [ OK ] Reached target Swap. [ OK ] Reached target Local File Systems. [ OK ] Listening on Journal Socket. Starting Apply Kernel Variables... Starting Setup Virtual Console... [ OK ] Started Memstrack Anylazing Service. Starting Journal Service... Starting Create list of required st…ce nodes for the current kernel... Starting Create Volatile Files and Directories... [ OK ] Started Dispatch Password Requests to Console Directory Watch. [ OK ] Reached target Local Encrypted Volumes. [ OK ] Reached target Paths. [ OK ] Listening on udev Control Socket. [ OK ] Reached target Sockets. [ OK ] Started Apply Kernel Variables. [ OK ] Started Setup Virtual Console. [ OK ] Started Create list of required sta…vice nodes for the current kernel. [ OK ] Started Create Volatile Files and Directories. Starting Create Static Device Nodes in /dev... Starting dracut cmdline hook... [ OK ] Started Create Static Device Nodes in /dev. [ OK ] Started Journal Service. [ OK ] Started dracut cmdline hook. Starting dracut pre-udev hook... [ 4.661626] device-mapper: uevent: version 1.0.3 [ 4.663203] device-mapper: ioctl: 4.46.0-ioctl (2022-02-22) initialised: dm-devel@redhat.com [ OK ] Started dracut pre-udev hook. Starting udev Kernel Device Manager... [ OK ] Started udev Kernel Device Manager. Starting dracut pre-trigger hook... [ OK ] Started dracut pre-trigger hook. Starting udev Coldplug all Devices... Mounting Kernel Configuration File System... [ OK ] Mounted Kernel Configuration File System. [ OK ] Started udev Coldplug all Devices. Starting dracut initqueue hook... [ OK ] Reached target System Initialization. [ OK ] Reached target Basic System. [ OK ] Started Hardware RNG Entropy Gatherer Daemon. [ 5.428953] random: fast init done [ 5.433752] virtio_net virtio0 ens2: renamed from eth0 [ 5.516585] scsi host0: ata_piix [ 5.528761] scsi host1: ata_piix [ 5.530058] ata1: PATA max MWDMA2 cmd 0x1f0 ctl 0x3f6 bmdma 0xc320 irq 14 [ 5.531946] ata2: PATA max MWDMA2 cmd 0x170 ctl 0x376 bmdma 0xc328 irq 15 [ 10.019322] dracut-initqueue[586]: RTNETLINK answers: File exists [ 10.256828] random: crng init done [ 10.258409] random: 7 urandom warning(s) missed due to ratelimiting Starting nbd nbd0... [ OK ] Started nbd nbd0. [ OK ] Started dracut initqueue hook. Mounting /sysroot... [ OK ] Reached target Remote File Systems (Pre). [ OK ] Reached target Remote File Systems. [ 10.746534] EXT4-fs (nbd0): mounted filesystem with ordered data mode. Opts: (null) [ OK ] Mounted /sysroot. [ OK ] Reached target Initrd Root File System. Starting Reload Configuration from the Real Root... [ OK ] Started Reload Configuration from the Real Root. [ OK ] Reached target Initrd File Systems. [ OK ] Reached target Initrd Default Target. Starting dracut pre-pivot and cleanup hook... [ OK ] Started dracut pre-pivot and cleanup hook. Starting Cleaning Up and Shutting Down Daemons... [ OK ] Stopped dracut pre-pivot and cleanup hook. [ OK ] Stopped target Remote File Systems. [ OK ] Stopped target Initrd Default Target. [ OK ] Stopped target Timers. [ OK ] Stopped target Initrd Root Device. [ OK ] Stopped target Remote File Systems (Pre). [ OK ] Stopped dracut initqueue hook. Stopping Hardware RNG Entropy Gatherer Daemon... [ OK ] Stopped Hardware RNG Entropy Gatherer Daemon. [ OK ] Started Cleaning Up and Shutting Down Daemons. [ OK ] Stopped target Basic System. [ OK ] Stopped target Paths. [ OK ] Stopped target Sockets. [ OK ] Stopped target Slices. [ OK ] Stopped target System Initialization. [ OK ] Stopped target Local Encrypted Volumes. [ OK ] Stopped Dispatch Password Requests to Console Directory Watch. [ OK ] Stopped udev Coldplug all Devices. [ OK ] Stopped dracut pre-trigger hook. [ OK ] Stopped target Swap. Stopping udev Kernel Device Manager... [ OK ] Stopped Apply Kernel Variables. [ OK ] Stopped Create Volatile Files and Directories. [ OK ] Stopped target Local File Systems. [ OK ] Stopped udev Kernel Device Manager. [ OK ] Stopped dracut pre-udev hook. [ OK ] Stopped dracut cmdline hook. [ OK ] Stopped Create Static Device Nodes in /dev. [ OK ] Stopped Create list of required sta…vice nodes for the current kernel. [ OK ] Closed udev Control Socket. [ OK ] Closed udev Kernel Socket. Starting Cleanup udevd DB... [ OK ] Started Cleanup udevd DB. [ OK ] Reached target Switch Root. Starting Switch Root... [ 11.895451] printk: systemd: 26 output lines suppressed due to ratelimiting [ 12.157201] SELinux: Disabled at runtime. [ 12.212687] systemd[1]: systemd 239 (239-82.el8_10.5) running in system mode. (+PAM +AUDIT +SELINUX +IMA -APPARMOR +SMACK +SYSVINIT +UTMP +LIBCRYPTSETUP +GCRYPT +GNUTLS +ACL +XZ +LZ4 +SECCOMP +BLKID +ELFUTILS +KMOD +IDN2 -IDN +PCRE2 default-hierarchy=legacy) [ 12.218491] systemd[1]: Detected virtualization kvm. [ 12.219740] systemd[1]: Detected architecture x86-64. Welcome to Rocky Linux 8.10 (Green Obsidian)! [ 12.900417] systemd[1]: initrd-switch-root.service: Succeeded. [ 12.903566] systemd[1]: Stopped Switch Root. [ OK ] Stopped Switch Root. [ 12.907549] systemd[1]: systemd-journald.service: Service has no hold-off time (RestartSec=0), scheduling restart. [ 12.912412] systemd[1]: systemd-journald.service: Scheduled restart job, restart counter is at 1. [ 12.915197] systemd[1]: Stopped Journal Service. [ OK ] Stopped Journal Service. [ 12.922324] systemd[1]: Starting Journal Service... Starting Journal Service... [ 12.939216] systemd[1]: Listening on Process Core Dump Socket. [ OK ] Listening on Process Core Dump Socket. [ OK ] Started Dispatch Password Requests to Console Directory Watch. [ OK ] Created slice system-getty.slice. [ OK ] Created slice User and Session Slice. [ OK ] Listening on initctl Compatibility Named Pipe. Mounting POSIX Message Queue File System... Starting Remount Root and Kernel File Systems... [ OK ] Reached target rpc_pipefs.target. Starting Create list of required st…ce nodes for the current kernel... [ OK ] Created slice system-serial\x2dgetty.slice. [ OK ] Listening on udev Control Socket. [ OK ] Listening on RPCbind Server Activation Socket. [ OK ] Listening on udev Kernel Socket. [ OK ] Reached target RPC Port Mapper. [ OK ] Reached target Slices. Mounting Kernel Debug File System... Starting udev Coldplug all Devices... Starting Apply Kernel Variables... Activating swap /dev/disk/by-label/SWAP... Mounting Huge Pages File System... [ OK ] Stopped target Switch Root. [ OK ] Stopped target Initrd File Systems. [ OK ] Stopped target Initrd Root File System. [ OK ] Started Forward Password Requests to Wall Directory Watch. [ OK ] Reached target Local Encrypted Volumes. [ OK ] Reached target Paths. [FAILED] Failed to set up automount Arbitrar…rmats File System Automount Point. See 'systemctl status proc-sys-fs-binfmt_misc.automount' for details. [ 13.093856] Adding 1048572k swap on /dev/vdb. Priority:-2 extents:1 across:1048572k FS [ OK ] Created slice system-sshd\x2dkeygen.slice. [ OK ] Started Journal Service. [ OK ] Mounted POSIX Message Queue File System. [FAILED] Failed to start Remount Root and Kernel File Systems. See 'systemctl status systemd-remount-fs.service' for details. [ OK ] Started Create list of required sta…vice nodes for the current kernel. [ OK ] Mounted Kernel Debug File System. [ OK ] Started Apply Kernel Variables. [ OK ] Activated swap /dev/disk/by-label/SWAP. [ OK ] Mounted Huge Pages File System. [ OK ] Reached target Swap. Starting Configure read-only root support... Starting Create Static Device Nodes in /dev... Starting Flush Journal to Persistent Storage... [ OK ] Started Flush Journal to Persistent Storage. [ OK ] Started Create Static Device Nodes in /dev. [ OK ] Reached target Local File Systems (Pre). Mounting /home/green/git/lustre-release... Mounting /mnt... Starting udev Kernel Device Manager... [ OK ] Mounted /mnt. [ OK ] Started udev Coldplug all Devices. [ 13.394381] squashfs: version 4.0 (2009/01/31) Phillip Lougher [ OK ] Mounted /home/green/git/lustre-release. [ OK ] Started udev Kernel Device Manager. [ 13.750707] piix4_smbus 0000:00:01.3: SMBus Host Controller at 0x700, revision 0 [ 13.798798] input: PC Speaker as /devices/platform/pcspkr/input/input5 [ 13.896360] RAPL PMU: API unit is 2^-32 Joules, 0 fixed counters, 10737418240 ms ovfl timer [ 13.917945] EDAC sbridge: Ver: 1.1.2 [ 15.588953] Key type dns_resolver registered [ 15.884776] NFS: Registering the id_resolver key type [ 15.886862] Key type id_resolver registered [ 15.888501] Key type id_legacy registered [ OK ] Started Configure read-only root support. [ OK ] Reached target Local File Systems. Starting Rebuild Dynamic Linker Cache... Starting Mark the need to relabel after reboot... Starting Create Volatile Files and Directories... Starting Load/Save Random Seed... [ OK ] Started Mark the need to relabel after reboot. [ OK ] Started Load/Save Random Seed. [ OK ] Started Create Volatile Files and Directories. Starting Update UTMP about System Boot/Shutdown... Starting RPC Bind... [ OK ] Started Update UTMP about System Boot/Shutdown. [ OK ] Started RPC Bind. [ OK ] Started Rebuild Dynamic Linker Cache. Starting Update is Completed... [ OK ] Started Update is Completed. [ OK ] Reached target System Initialization. [ OK ] Started Daily Cleanup of Temporary Directories. [ OK ] Started dnf makecache --timer. [ OK ] Started daily update of the root trust anchor for DNSSEC. [ OK ] Reached target Timers. [ OK ] Listening on D-Bus System Message Bus Socket. [ OK ] Reached target Sockets. [ OK ] Reached target Basic System. [ OK ] Started Hardware RNG Entropy Gatherer Daemon. Starting Restore /run/initramfs on shutdown... [ OK ] Started D-Bus System Message Bus. Starting Network Manager... [ OK ] Started irqbalance daemon. [ OK ] Reached target sshd-keygen.target. Starting Login Service... [ OK ] Started Restore /run/initramfs on shutdown. [ OK ] Started Login Service. [ OK ] Started Network Manager. [ OK ] Reached target Network. Starting OpenSSH server daemon... Starting Dynamic System Tuning Daemon... Starting GSSAPI Proxy Daemon... Starting Network Manager Wait Online... [ OK ] Started OpenSSH server daemon. Starting Hostname Service... [ OK ] Started GSSAPI Proxy Daemon. [ OK ] Reached target NFS client services. [ OK ] Reached target Remote File Systems (Pre). [ OK ] Reached target Remote File Systems. Starting Permit User Sessions... [ OK ] Started Permit User Sessions. [ OK ] Started Serial Getty on ttyS0. [ OK ] Started Getty on tty1. [ OK ] Started Serial Getty on ttyS1. [ OK ] Reached target Login Prompts. [ OK ] Started Command Scheduler. [ OK ] Started Hostname Service. Starting Network Manager Script Dispatcher Service... [ OK ] Started Network Manager Script Dispatcher Service. [ OK ] Started Network Manager Wait Online. [ OK ] Reached target Network is Online. Starting Notify NFS peers of a restart... Starting Crash recovery kernel arming... Starting System Logging Service... [ OK ] Started Notify NFS peers of a restart. [ OK ] Started System Logging Service. Starting Authorization Manager... [ OK ] Started Authorization Manager. [ OK ] Started Dynamic System Tuning Daemon. [ OK ] Reached target Multi-User System. [ OK ] Reached target Graphical Interface. Starting Update UTMP about System Runlevel Changes... [ OK ] Started Update UTMP about System Runlevel Changes. [ OK ] Started Crash recovery kernel arming. Rocky Linux 8.10 (Green Obsidian) Kernel 4.18.0rh8.10-debug on an x86_64 oleg234-server login: [ 41.802193] libcfs: loading out-of-tree module taints kernel. [ 41.823135] Key type ._llcrypt registered [ 41.824994] Key type .llcrypt registered [ 41.878924] Lustre: DEBUG MARKER: oleg234-server.virtnet: executing set_hostid [ 55.864979] Lustre: DEBUG MARKER: oleg234-server.virtnet: executing load_modules_local [ 57.401819] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 57.424807] alg: No test for adler32 (adler32-zlib) [ 58.980729] Lustre: Lustre: Build Version: 2.17.50_65_g69bb343 [ 60.079835] LNet: Added LNI 192.168.202.134@tcp [8/256/0/180] [ 61.839180] Key type lgssc registered [ 64.172921] Lustre: Echo OBD driver; http://www.lustre.org/ [ 86.272032] hrtimer: interrupt took 6021612 ns [ 88.226306] ZFS: Loaded module v2.3.2-1, ZFS pool version 5000, ZFS filesystem version 5 [ 141.338573] Lustre: DEBUG MARKER: oleg234-server.virtnet: executing load_modules_local [ 156.175350] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 156.237723] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 157.510131] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 157.532204] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 157.586216] Lustre: lustre-MDT0000: new disk, initializing [ 157.644760] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 157.655990] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 162.348456] Lustre: DEBUG MARKER: oleg234-server.virtnet: executing set_default_debug vfstrace rpctrace dlmtrace neterror ha config ioctl super lfsck all [ 178.988130] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 179.096519] Lustre: 6485:0:(mgs_llog.c:1348:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 179.124490] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [ 179.128060] Lustre: Skipped 1 previous similar message [ 179.237288] Lustre: lustre-MDT0001: new disk, initializing [ 179.334526] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 179.370755] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [ 179.388858] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 184.746692] Lustre: DEBUG MARKER: oleg234-server.virtnet: executing set_default_debug vfstrace rpctrace dlmtrace neterror ha config ioctl super lfsck all [ 190.619231] Lustre: Modifying parameter general.debug_raw_pointers=Y in log params [ 202.566337] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 202.940433] Lustre: lustre-OST0000: new disk, initializing [ 202.947294] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 202.956991] Lustre: 8389:0:(osd_compat.c:1354:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 203.034881] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 209.431978] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [ 209.447074] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 209.542604] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 210.262456] Lustre: DEBUG MARKER: oleg234-server.virtnet: executing set_default_debug vfstrace rpctrace dlmtrace neterror ha config ioctl super lfsck all [ 227.438556] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 227.624715] Lustre: lustre-OST0001: new disk, initializing [ 227.631711] Lustre: srv-lustre-OST0001: No data found on store. Initialize space. [ 227.653470] Lustre: 9444:0:(osd_compat.c:1354:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 227.772119] Lustre: lustre-OST0001: Imperative Recovery not enabled, recovery window 60-180 [ 236.057690] Lustre: DEBUG MARKER: oleg234-server.virtnet: executing set_default_debug vfstrace rpctrace dlmtrace neterror ha config ioctl super lfsck all [ 237.606146] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x00000002c0000400-0x0000000300000400]:1:ost [ 237.613585] Lustre: cli-lustre-OST0001-super: Allocated super-sequence [0x00000002c0000400-0x0000000300000400]:1:ost] [ 237.700494] Lustre: lustre-OST0001-osc-MDT0000: update sequence from 0x100010000 to 0x2c0000401 [ 250.562156] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 258.588803] Lustre: Setting parameter general.lod.*.mdt_hash=crush in log params [ 272.145878] Lustre: DEBUG MARKER: oleg234-server.virtnet: executing check_logdir /tmp/testlogs/ [ 278.242859] Lustre: DEBUG MARKER: oleg234-server.virtnet: executing yml_node [ 283.368402] Lustre: DEBUG MARKER: Client: 2.17.50.65 [ 286.904475] Lustre: DEBUG MARKER: MDS: 2.17.50.65 [ 289.794970] Lustre: DEBUG MARKER: OSS: 2.17.50.65 [ 292.037882] Lustre: DEBUG MARKER: -----============= acceptance-small: conf-sanity ============----- Mon Feb 9 19:57:25 EST 2026 [ 315.850271] Lustre: DEBUG MARKER: excepting tests: 102 106 115 32newtarball [ 317.747153] Lustre: DEBUG MARKER: skipping tests SLOW=no: 45 69 106 111 114 [ 334.823120] Lustre: lustre-MDT0000-lwp-OST0000: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 334.834517] Lustre: Skipped 2 previous similar messages [ 334.835664] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 334.855095] Lustre: Skipped 3 previous similar messages [ 339.948628] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 339.957770] Lustre: Skipped 3 previous similar messages [ 345.063458] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 345.075195] Lustre: Skipped 3 previous similar messages [ 349.151150] Lustre: lustre-MDT0000 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 349.281898] LustreError: 12184:0:(obd_class.h:479:obd_check_dev()) Device 28 not setup [ 349.497868] Lustre: server umount lustre-MDT0000 complete [ 353.931734] LustreError: 6477:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1770685109 with bad export cookie 5248568070740601165 [ 353.936554] LustreError: MGC192.168.202.134@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 353.944678] LustreError: 6477:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 355.298313] LustreError: 6491:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 355.298817] Lustre: lustre-MDT0001-lwp-OST0000: Connection to lustre-MDT0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 355.314422] Lustre: lustre-MDT0001: Not available for connect from 0@lo (stopping) [ 355.326964] LustreError: 6491:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 6 previous similar messages [ 355.367312] Lustre: Skipped 2 previous similar messages [ 358.319192] LustreError: 6493:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 358.347718] LustreError: 6493:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 1 previous similar message [ 360.183386] LustreError: 12387:0:(obd_class.h:479:obd_check_dev()) Device 27 not setup [ 360.189916] LustreError: 12387:0:(obd_class.h:479:obd_check_dev()) Skipped 7 previous similar messages [ 360.337376] Lustre: server umount lustre-MDT0001 complete [ 364.191207] LustreError: 12589:0:(obd_class.h:479:obd_check_dev()) Device 19 not setup [ 364.194209] LustreError: 12589:0:(obd_class.h:479:obd_check_dev()) Skipped 7 previous similar messages [ 364.254314] Lustre: server umount lustre-OST0000 complete [ 368.159165] LustreError: 12790:0:(obd_class.h:479:obd_check_dev()) Device 2 not setup [ 368.162321] LustreError: 12790:0:(obd_class.h:479:obd_check_dev()) Skipped 3 previous similar messages [ 368.327286] Lustre: server umount lustre-OST0001 complete [ 376.241638] Lustre: DEBUG MARKER: oleg234-server.virtnet: executing set_hostid [ 386.185700] Lustre: DEBUG MARKER: oleg234-server.virtnet: executing load_modules_local [ 429.473250] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 429.858707] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 429.939443] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 430.055173] Lustre: lustre-MDT0000: new disk, initializing [ 430.132136] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 430.148278] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 435.469364] Lustre: DEBUG MARKER: oleg234-server.virtnet: executing set_default_debug -1 all [ 447.471519] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 447.671120] Lustre: 14964:0:(mgs_llog.c:1348:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 447.791390] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [ 447.796385] Lustre: Skipped 1 previous similar message [ 447.958312] Lustre: lustre-MDT0001: new disk, initializing [ 448.101982] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 448.184287] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [ 448.198043] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 453.545973] Lustre: DEBUG MARKER: oleg234-server.virtnet: executing set_default_debug -1 all [ 462.007612] Lustre: DEBUG MARKER: oleg234-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 466.490315] Lustre: DEBUG MARKER: oleg234-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 475.404604] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 475.840659] Lustre: lustre-OST0000: new disk, initializing [ 475.844903] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 475.856535] Lustre: 16707:0:(osd_compat.c:1354:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 475.953782] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 478.009814] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [ 478.021621] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 478.255816] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 483.328731] Lustre: DEBUG MARKER: oleg234-server.virtnet: executing set_default_debug -1 all [ 492.230725] Lustre: DEBUG MARKER: oleg234-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 497.519570] Lustre: DEBUG MARKER: oleg234-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 [ 497.754849] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [ 503.150364] Lustre: DEBUG MARKER: oleg234-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid 50 [ 503.411951] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [ 508.901317] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 508.902675] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 508.927900] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 508.946786] Lustre: Skipped 4 previous similar messages [ 511.821378] LustreError: 17900:0:(obd_class.h:479:obd_check_dev()) Device 19 not setup [ 511.833322] LustreError: 17900:0:(obd_class.h:479:obd_check_dev()) Skipped 5 previous similar messages [ 512.251730] Lustre: server umount lustre-OST0000 complete [ 519.137167] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 519.172018] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 519.185381] Lustre: Skipped 2 previous similar messages [ 519.209563] Lustre: Skipped 1 previous similar message [ 522.883468] LustreError: 18114:0:(obd_class.h:479:obd_check_dev()) Device 22 not setup [ 522.907651] LustreError: 18114:0:(obd_class.h:479:obd_check_dev()) Skipped 3 previous similar messages [ 523.125742] Lustre: server umount lustre-MDT0000 complete [ 524.267526] LustreError: 14975:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 524.290859] LustreError: 14975:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 1 previous similar message [ 528.006229] LustreError: 14955:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1770685283 with bad export cookie 5248568070740602705 [ 528.016247] LustreError: MGC192.168.202.134@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 528.025765] LustreError: 14955:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 528.420841] Lustre: server umount lustre-MDT0001 complete [ 534.682534] Lustre: DEBUG MARKER: == conf-sanity test 76a: set permanent params with lctl across mounts ========================================================== 20:01:28 (1770685288) [ 543.750483] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 544.257887] LustreError: 18843:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 544.371539] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 549.197374] Lustre: DEBUG MARKER: oleg234-server.virtnet: executing set_default_debug -1 all [ 549.865444] LustreError: 18843:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 559.540056] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 564.758657] Lustre: DEBUG MARKER: oleg234-server.virtnet: executing set_default_debug -1 all [ 570.403708] Lustre: DEBUG MARKER: oleg234-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 574.443391] Lustre: DEBUG MARKER: oleg234-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 582.272736] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 589.292572] Lustre: DEBUG MARKER: oleg234-server.virtnet: executing set_default_debug -1 all [ 596.776859] Lustre: DEBUG MARKER: oleg234-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 601.239321] Lustre: Modifying parameter general.osc.*.max_dirty_mb=452 in log params [ 610.279397] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 610.296016] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 610.308872] Lustre: Skipped 2 previous similar messages [ 615.558530] LustreError: 21140:0:(obd_class.h:479:obd_check_dev()) Device 9 not setup [ 615.577044] LustreError: 21140:0:(obd_class.h:479:obd_check_dev()) Skipped 13 previous similar messages [ 615.854092] Lustre: server umount lustre-MDT0000 complete [ 620.467053] LustreError: 21141:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1770685376 with bad export cookie 5248568070740603587 [ 620.468806] LustreError: MGC192.168.202.134@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 620.487737] LustreError: 21141:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) Skipped 1 previous similar message [ 620.539639] Lustre: lustre-MDT0001-lwp-OST0000: Connection to lustre-MDT0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 620.542504] LustreError: 19817:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 620.565563] Lustre: Skipped 2 previous similar messages [ 620.616604] LustreError: 19817:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 1 previous similar message [ 620.958843] Lustre: server umount lustre-MDT0001 complete [ 627.614696] Lustre: server umount lustre-OST0000 complete [ 650.288911] Lustre: DEBUG MARKER: oleg234-server.virtnet: executing load_modules_local [ 663.204513] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 663.681095] LustreError: 22871:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 663.763085] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 663.766147] Lustre: Skipped 2 previous similar messages [ 668.737126] Lustre: DEBUG MARKER: oleg234-server.virtnet: executing set_default_debug -1 all [ 679.274956] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 685.306745] Lustre: DEBUG MARKER: oleg234-server.virtnet: executing set_default_debug -1 all [ 688.607580] Lustre: Setting parameter general.debug_raw_pointers=Y in log params [ 695.527631] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 702.992330] Lustre: DEBUG MARKER: oleg234-server.virtnet: executing set_default_debug -1 all [ 713.107668] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 713.350611] Lustre: lustre-OST0001: new disk, initializing [ 713.360493] Lustre: srv-lustre-OST0001: No data found on store. Initialize space. [ 713.374567] Lustre: 25141:0:(osd_compat.c:1354:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 715.249564] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x00000002c0000400-0x0000000300000400]:1:ost [ 715.261574] Lustre: cli-lustre-OST0001-super: Allocated super-sequence [0x00000002c0000400-0x0000000300000400]:1:ost] [ 715.411791] Lustre: lustre-OST0001-osc-MDT0000: update sequence from 0x100010000 to 0x2c0000401 [ 720.085876] Lustre: DEBUG MARKER: oleg234-server.virtnet: executing set_default_debug -1 all [ 731.607600] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 736.242406] Lustre: Setting parameter general.lod.*.mdt_hash=crush in log params [ 756.200761] Lustre: lustre-MDT0000-lwp-OST0000: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 756.214507] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 756.215844] Lustre: Skipped 2 previous similar messages [ 756.233295] Lustre: Skipped 7 previous similar messages [ 762.091714] LustreError: 26349:0:(obd_class.h:479:obd_check_dev()) Device 28 not setup [ 762.121134] LustreError: 26349:0:(obd_class.h:479:obd_check_dev()) Skipped 17 previous similar messages [ 762.278463] Lustre: server umount lustre-MDT0000 complete [ 766.439692] LustreError: 22871:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 766.456432] LustreError: 22871:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 6 previous similar messages [ 766.641651] LustreError: 26094:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1770685522 with bad export cookie 5248568070740604434 [ 766.648968] LustreError: MGC192.168.202.134@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 766.657620] LustreError: 26094:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 767.283888] Lustre: server umount lustre-MDT0001 complete [ 782.144168] Lustre: server umount lustre-OST0000 complete [ 796.575518] Lustre: server umount lustre-OST0001 complete [ 816.958260] Lustre: DEBUG MARKER: oleg234-server.virtnet: executing load_modules_local [ 828.133534] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 828.656105] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 828.658842] Lustre: Skipped 3 previous similar messages [ 833.522368] Lustre: DEBUG MARKER: oleg234-server.virtnet: executing set_default_debug -1 all [ 834.020869] LustreError: 28129:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 834.039210] LustreError: 28129:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 1 previous similar message [ 843.784384] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 849.109646] Lustre: DEBUG MARKER: oleg234-server.virtnet: executing set_default_debug -1 all [ 852.841709] Lustre: 29242:0:(mgs_llog.c:1348:mgs_modify_param()) MGS: modify general/debug_raw_pointers=Y (mode = 0) failed: rc = -17 [ 859.961845] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 867.713847] Lustre: DEBUG MARKER: oleg234-server.virtnet: executing set_default_debug -1 all [ 877.186806] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 883.202473] Lustre: DEBUG MARKER: oleg234-server.virtnet: executing set_default_debug -1 all [ 891.154971] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 894.645112] Lustre: 31094:0:(mgs_llog.c:1348:mgs_modify_param()) MGS: modify general/lod.*.mdt_hash=crush (mode = 0) failed: rc = -17 [ 908.273254] Lustre: lustre-MDT0000-lwp-OST0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 908.286061] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 908.293625] Lustre: Skipped 4 previous similar messages [ 908.311960] Lustre: Skipped 7 previous similar messages [ 912.623192] LustreError: 31299:0:(obd_class.h:479:obd_check_dev()) Device 10 not setup [ 912.627337] LustreError: 31299:0:(obd_class.h:479:obd_check_dev()) Skipped 25 previous similar messages [ 912.799583] Lustre: server umount lustre-MDT0000 complete [ 916.099893] LustreError: 28109:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1770685671 with bad export cookie 5248568070740605729 [ 916.115185] LustreError: 28109:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) Skipped 2 previous similar messages [ 916.116695] LustreError: MGC192.168.202.134@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 916.416916] Lustre: server umount lustre-MDT0001 complete [ 929.539349] Lustre: server umount lustre-OST0000 complete [ 943.693169] Lustre: server umount lustre-OST0001 complete [ 953.413592] Lustre: DEBUG MARKER: == conf-sanity test 76b: verify params log setup correctly ========================================================== 20:08:27 (1770685707) [ 984.624765] Lustre: DEBUG MARKER: oleg234-server.virtnet: executing load_modules_local [ 993.834358] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 994.234781] LustreError: 33882:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 994.254598] LustreError: 33882:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 8 previous similar messages [ 998.153655] Lustre: DEBUG MARKER: oleg234-server.virtnet: executing set_default_debug -1 all [ 1006.787745] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 1011.489220] Lustre: DEBUG MARKER: oleg234-server.virtnet: executing set_default_debug -1 all [ 1014.487881] Lustre: 34997:0:(mgs_llog.c:1348:mgs_modify_param()) MGS: modify general/debug_raw_pointers=Y (mode = 0) failed: rc = -17 [ 1021.452639] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 1026.823746] Lustre: DEBUG MARKER: oleg234-server.virtnet: executing set_default_debug -1 all [ 1034.884098] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 1042.129730] Lustre: DEBUG MARKER: oleg234-server.virtnet: executing set_default_debug -1 all [ 1049.924629] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 1053.794782] Lustre: 36849:0:(mgs_llog.c:1348:mgs_modify_param()) MGS: modify general/lod.*.mdt_hash=crush (mode = 0) failed: rc = -17 [ 1065.441508] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 1065.459492] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 1065.472198] Lustre: Skipped 2 previous similar messages [ 1065.488266] Lustre: Skipped 3 previous similar messages [ 1070.840601] LustreError: 37001:0:(obd_class.h:479:obd_check_dev()) Device 10 not setup [ 1070.852543] LustreError: 37001:0:(obd_class.h:479:obd_check_dev()) Skipped 25 previous similar messages [ 1071.043833] Lustre: server umount lustre-MDT0000 complete [ 1074.628141] LustreError: 33862:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1770685830 with bad export cookie 5248568070740606835 [ 1074.633499] LustreError: MGC192.168.202.134@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 1074.642014] LustreError: 33862:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) Skipped 1 previous similar message [ 1074.919509] Lustre: server umount lustre-MDT0001 complete [ 1088.747059] Lustre: server umount lustre-OST0000 complete [ 1092.069317] Lustre: 3633:0:(client.c:2478:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1770685831/real 1770685831] req@ffff9c1405a51180 x1856697600941312/t0(0) o400->lustre-MDT0001-lwp-OST0001@0@lo:12/10 lens 224/224 e 0 to 1 dl 1770685847 ref 1 fl Rpc:XNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 [ 1092.244655] Lustre: server umount lustre-OST0001 complete [ 1101.228390] Lustre: DEBUG MARKER: == conf-sanity test 76c: verify changelog_mask is applied with lctl set_param -P ========================================================== 20:10:55 (1770685855) [ 1119.709860] Lustre: DEBUG MARKER: oleg234-server.virtnet: executing load_modules_local [ 1130.741892] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1131.167330] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 1131.170430] Lustre: Skipped 7 previous similar messages [ 1135.161328] Lustre: DEBUG MARKER: oleg234-server.virtnet: executing set_default_debug -1 all [ 1143.953639] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 1148.605902] Lustre: DEBUG MARKER: oleg234-server.virtnet: executing set_default_debug -1 all [ 1151.445529] Lustre: 40087:0:(mgs_llog.c:1348:mgs_modify_param()) MGS: modify general/debug_raw_pointers=Y (mode = 0) failed: rc = -17 [ 1158.680450] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 1163.910883] Lustre: DEBUG MARKER: oleg234-server.virtnet: executing set_default_debug -1 all [ 1172.092814] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 1179.521729] Lustre: DEBUG MARKER: oleg234-server.virtnet: executing set_default_debug -1 all [ 1187.496719] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 1191.115070] Lustre: 41938:0:(mgs_llog.c:1348:mgs_modify_param()) MGS: modify general/lod.*.mdt_hash=crush (mode = 0) failed: rc = -17 [ 1197.140473] Lustre: Setting parameter general.mdd.*.changelog_mask=-CLOSE in log params [ 1197.149137] Lustre: Skipped 1 previous similar message [ 1200.613657] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 1200.624528] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 1200.638975] Lustre: Skipped 2 previous similar messages [ 1206.570557] Lustre: server umount lustre-MDT0000 complete [ 1210.075126] LustreError: 38954:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1770685965 with bad export cookie 5248568070740607941 [ 1210.086519] LustreError: MGC192.168.202.134@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 1210.093225] LustreError: 38954:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 1216.386790] Lustre: server umount lustre-MDT0001 complete [ 1224.180303] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1234.399874] LustreError: 3632:0:(client.c:1390:ptlrpc_import_delay_req()) @@@ invalidate in flight req@ffff9c142f3fc000 x1856697600996608/t0(0) o250->MGC192.168.202.134@tcp@0@lo:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl Rpc:NQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 [ 1238.800632] Lustre: DEBUG MARKER: oleg234-server.virtnet: executing set_default_debug -1 all [ 1245.101581] Lustre: lustre-MDT0000-lwp-OST0000: Connection restored to 0@lo (at 0@lo) [ 1246.809801] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 1251.186941] Lustre: DEBUG MARKER: oleg234-server.virtnet: executing set_default_debug -1 all [ 1252.356308] Lustre: lustre-MDT0001-lwp-OST0000: Connection restored to 0@lo (at 0@lo) [ 1252.364889] Lustre: Skipped 1 previous similar message [ 1257.035783] Lustre: DEBUG MARKER: oleg234-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 1258.566963] Lustre: DEBUG MARKER: mdc.lustre-MDT0000-mdc-*.mds_server_uuid in FULL state after 0 sec [ 1262.679270] Lustre: DEBUG MARKER: oleg234-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 1264.659237] Lustre: DEBUG MARKER: mdc.lustre-MDT0001-mdc-*.mds_server_uuid in FULL state after 0 sec [ 1272.806630] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 1276.645069] Lustre: server umount lustre-MDT0000 complete [ 1277.931908] LustreError: 42816:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 1277.966102] LustreError: 42816:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 28 previous similar messages [ 1280.931718] LustreError: 38955:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1770686036 with bad export cookie 5248568070740609103 [ 1280.933906] LustreError: MGC192.168.202.134@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 1280.947568] LustreError: 38955:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 1281.381126] Lustre: server umount lustre-MDT0001 complete [ 1295.745157] Lustre: server umount lustre-OST0000 complete [ 1308.488200] Lustre: server umount lustre-OST0001 complete [ 1316.173365] Lustre: DEBUG MARKER: == conf-sanity test 76d: verify llite.*.xattr_cache can be set by 'lctl set_param -P' correctly ========================================================== 20:14:30 (1770686070) [ 1331.189920] Lustre: DEBUG MARKER: oleg234-server.virtnet: executing load_modules_local [ 1339.246033] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1343.517265] Lustre: DEBUG MARKER: oleg234-server.virtnet: executing set_default_debug -1 all [ 1351.861610] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 1356.160564] Lustre: DEBUG MARKER: oleg234-server.virtnet: executing set_default_debug -1 all [ 1358.467350] Lustre: 47424:0:(mgs_llog.c:1348:mgs_modify_param()) MGS: modify general/debug_raw_pointers=Y (mode = 0) failed: rc = -17 [ 1363.662716] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 1369.385054] Lustre: DEBUG MARKER: oleg234-server.virtnet: executing set_default_debug -1 all [ 1377.608156] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 1382.812849] Lustre: DEBUG MARKER: oleg234-server.virtnet: executing set_default_debug -1 all [ 1389.598238] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 1398.877844] Lustre: Setting parameter general.llite.*.xattr_cache=0 in log params [ 1413.081331] LustreError: 49496:0:(obd_class.h:479:obd_check_dev()) Device 10 not setup [ 1413.086653] LustreError: 49496:0:(obd_class.h:479:obd_check_dev()) Skipped 67 previous similar messages [ 1413.333963] Lustre: server umount lustre-MDT0000 complete [ 1413.622213] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 1417.699745] LustreError: 47426:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1770686173 with bad export cookie 5248568070740609908 [ 1417.706181] LustreError: MGC192.168.202.134@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 1417.713355] LustreError: 47426:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 1418.012521] Lustre: server umount lustre-MDT0001 complete [ 1431.792549] Lustre: server umount lustre-OST0000 complete [ 1434.783167] Lustre: 3636:0:(client.c:2478:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1770686174/real 1770686174] req@ffff9c1303ad8a80 x1856697601088512/t0(0) o400->lustre-MDT0001-lwp-OST0001@0@lo:12/10 lens 224/224 e 0 to 1 dl 1770686190 ref 1 fl Rpc:XNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 [ 1435.223113] Lustre: server umount lustre-OST0001 complete [ 1443.772464] Lustre: DEBUG MARKER: == conf-sanity test 77: comma-separated MGS NIDs and failover node NIDs ========================================================== 20:16:38 (1770686198) [ 1445.690681] Lustre: DEBUG MARKER: SKIP: conf-sanity test_77 mixed loopback and real device not working [ 1447.276439] Lustre: DEBUG MARKER: == conf-sanity test 78: run resize2fs on MDT and OST filesystems ========================================================== 20:16:41 (1770686201) [ 1455.801954] Lustre: DEBUG MARKER: oleg234-server.virtnet: executing set_hostid [ 1464.432545] Lustre: DEBUG MARKER: oleg234-server.virtnet: executing load_modules_local [ 1482.279599] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1482.539193] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 1482.645248] Lustre: lustre-MDT0000: new disk, initializing [ 1482.725196] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 1487.108694] Lustre: DEBUG MARKER: oleg234-server.virtnet: executing set_default_debug -1 all [ 1494.436780] Lustre: DEBUG MARKER: oleg234-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 1501.289349] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 1501.647452] Lustre: lustre-OST0000: new disk, initializing [ 1501.650708] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 1501.659697] Lustre: Skipped 1 previous similar message [ 1501.662520] Lustre: 53324:0:(osd_compat.c:1354:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 1503.484481] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:0:ost [ 1503.494606] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:0:ost] [ 1503.541916] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x240000400 [ 1507.508486] Lustre: DEBUG MARKER: oleg234-server.virtnet: executing set_default_debug -1 all [ 1514.304556] Lustre: DEBUG MARKER: oleg234-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 1516.269826] Lustre: DEBUG MARKER: create test files [ 1544.674663] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 1544.679524] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 1544.704107] Lustre: Skipped 14 previous similar messages [ 1544.721550] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 1544.729501] Lustre: Skipped 19 previous similar messages [ 1547.944192] Lustre: server umount lustre-OST0000 complete [ 1551.675699] Lustre: server umount lustre-MDT0000 complete [ 1561.554932] Lustre: DEBUG MARKER: oleg234-server.virtnet: executing unload_modules_local [ 1563.946788] Key type lgssc unregistered [ 1564.208482] LNet: 54829:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 1564.220404] LNetError: 54829:0:(acceptor.c:264:lnet_acceptor_remove_socket()) Interface ens2 not found [ 1565.299543] LNet: Removed LNI 192.168.202.134@tcp [ 1566.111421] Key type .llcrypt unregistered [ 1566.115922] Key type ._llcrypt unregistered [ 1592.249158] Key type ._llcrypt registered [ 1592.255552] Key type .llcrypt registered [ 1592.372655] Lustre: DEBUG MARKER: oleg234-server.virtnet: executing load_modules_local [ 1593.759612] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 1593.807983] alg: No test for adler32 (adler32-zlib) [ 1594.767360] Lustre: Lustre: Build Version: 2.17.50_65_g69bb343 [ 1594.955176] LNet: Added LNI 192.168.202.134@tcp [8/256/0/180] [ 1596.607224] Key type lgssc registered [ 1597.930754] Lustre: Echo OBD driver; http://www.lustre.org/ [ 1604.272482] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 1604.295223] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1605.692801] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 1610.289645] Lustre: DEBUG MARKER: oleg234-server.virtnet: executing set_default_debug -1 all [ 1615.434939] Lustre: DEBUG MARKER: oleg234-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 1621.613620] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 1621.893965] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 1626.698688] Lustre: DEBUG MARKER: oleg234-server.virtnet: executing set_default_debug -1 all [ 1631.165137] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x240000400:97 to 0x240000400:129) [ 1631.298766] Lustre: 57431:0:(osd_internal.h:1459:osd_trans_exec_op()) lustre-OST0000: opcode 8: before 515 < left 544, rollback = 9 [ 1631.308370] Lustre: 57431:0:(osd_handler.c:2096:osd_trans_dump_creds()) create: 32/128/0, destroy: 0/0/0 [ 1631.318991] Lustre: 57431:0:(osd_handler.c:2103:osd_trans_dump_creds()) attr_set: 0/0/0, xattr_set: 0/0/0 [ 1631.332945] Lustre: 57431:0:(osd_handler.c:2113:osd_trans_dump_creds()) write: 2/13/1, punch: 0/0/0, quota 1/3/0 [ 1631.341613] Lustre: 57431:0:(osd_handler.c:2120:osd_trans_dump_creds()) insert: 32/544/0, delete: 0/0/0 [ 1631.347200] Lustre: 57431:0:(osd_handler.c:2127:osd_trans_dump_creds()) ref_add: 0/0/0, ref_del: 0/0/0 [ 1631.588662] Lustre: DEBUG MARKER: oleg234-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 1636.316374] Lustre: DEBUG MARKER: check files after expanding the MDT and OST filesystems [ 1641.500895] Lustre: DEBUG MARKER: create more files after expanding the MDT and OST filesystems [ 1649.820778] LustreError: 58178:0:(obd_class.h:479:obd_check_dev()) Device 13 not setup [ 1650.049309] Lustre: server umount lustre-OST0000 complete [ 1663.463964] LustreError: 58379:0:(obd_class.h:479:obd_check_dev()) Device 8 not setup [ 1663.474430] LustreError: 58379:0:(obd_class.h:479:obd_check_dev()) Skipped 1 previous similar message [ 1663.658670] Lustre: server umount lustre-MDT0000 complete [ 1672.950616] Lustre: DEBUG MARKER: oleg234-server.virtnet: executing unload_modules_local [ 1675.219509] Key type lgssc unregistered [ 1675.450684] LNet: 58912:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 1675.463310] LNetError: 58912:0:(acceptor.c:264:lnet_acceptor_remove_socket()) Interface ens2 not found [ 1675.489687] LNet: Removed LNI 192.168.202.134@tcp [ 1676.220457] Key type .llcrypt unregistered [ 1676.222651] Key type ._llcrypt unregistered [ 1703.569174] Key type ._llcrypt registered [ 1703.571567] Key type .llcrypt registered [ 1703.679229] Lustre: DEBUG MARKER: oleg234-server.virtnet: executing load_modules_local [ 1705.001105] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 1705.047263] alg: No test for adler32 (adler32-zlib) [ 1706.121875] Lustre: Lustre: Build Version: 2.17.50_65_g69bb343 [ 1706.307125] LNet: Added LNI 192.168.202.134@tcp [8/256/0/180] [ 1707.935525] Key type lgssc registered [ 1709.050853] Lustre: Echo OBD driver; http://www.lustre.org/ [ 1716.462430] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 1716.490635] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1718.000598] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 1721.986257] Lustre: DEBUG MARKER: oleg234-server.virtnet: executing set_default_debug -1 all [ 1727.063336] Lustre: DEBUG MARKER: oleg234-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 1733.366981] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 1733.666992] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 1738.695661] Lustre: DEBUG MARKER: oleg234-server.virtnet: executing set_default_debug -1 all [ 1742.896983] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x240000400:140 to 0x240000400:161) [ 1742.986229] Lustre: 61598:0:(osd_internal.h:1459:osd_trans_exec_op()) lustre-OST0000: opcode 8: before 515 < left 544, rollback = 9 [ 1743.000748] Lustre: 61598:0:(osd_handler.c:2096:osd_trans_dump_creds()) create: 32/128/0, destroy: 0/0/0 [ 1743.006501] Lustre: 61598:0:(osd_handler.c:2103:osd_trans_dump_creds()) attr_set: 0/0/0, xattr_set: 0/0/0 [ 1743.009597] Lustre: 61598:0:(osd_handler.c:2113:osd_trans_dump_creds()) write: 2/13/1, punch: 0/0/0, quota 1/3/0 [ 1743.013550] Lustre: 61598:0:(osd_handler.c:2120:osd_trans_dump_creds()) insert: 32/544/0, delete: 0/0/0 [ 1743.018850] Lustre: 61598:0:(osd_handler.c:2127:osd_trans_dump_creds()) ref_add: 0/0/0, ref_del: 0/0/0 [ 1743.930626] Lustre: DEBUG MARKER: oleg234-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 1749.250698] Lustre: DEBUG MARKER: check files after shrinking the MDT and OST filesystems [ 1758.184058] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 1758.189717] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 1758.202902] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 1763.297278] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 1763.599140] LustreError: 62291:0:(obd_class.h:479:obd_check_dev()) Device 13 not setup [ 1763.682966] Lustre: server umount lustre-OST0000 complete [ 1767.988163] LustreError: 62492:0:(obd_class.h:479:obd_check_dev()) Device 8 not setup [ 1767.994075] LustreError: 62492:0:(obd_class.h:479:obd_check_dev()) Skipped 1 previous similar message [ 1768.209644] Lustre: server umount lustre-MDT0000 complete [ 1778.342151] Lustre: DEBUG MARKER: oleg234-server.virtnet: executing unload_modules_local [ 1780.738433] Key type lgssc unregistered [ 1780.980318] LNet: 63025:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 1780.990831] LNetError: 63025:0:(acceptor.c:264:lnet_acceptor_remove_socket()) Interface ens2 not found [ 1782.056891] LNet: Removed LNI 192.168.202.134@tcp [ 1782.830793] Key type .llcrypt unregistered [ 1782.832944] Key type ._llcrypt unregistered [ 1798.433649] Key type ._llcrypt registered [ 1798.435898] Key type .llcrypt registered [ 1798.551840] Lustre: DEBUG MARKER: oleg234-server.virtnet: executing set_hostid [ 1813.024446] Lustre: DEBUG MARKER: oleg234-server.virtnet: executing load_modules_local [ 1814.086698] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 1814.208405] alg: No test for adler32 (adler32-zlib) [ 1815.180608] Lustre: Lustre: Build Version: 2.17.50_65_g69bb343 [ 1815.338791] LNet: Added LNI 192.168.202.134@tcp [8/256/0/180] [ 1816.975220] Key type lgssc registered [ 1817.764122] Lustre: Echo OBD driver; http://www.lustre.org/ [ 1846.127320] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 1846.148413] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1847.370234] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 1847.393109] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 1847.460053] Lustre: lustre-MDT0000: new disk, initializing [ 1847.551394] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 1847.588220] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 1851.155846] Lustre: DEBUG MARKER: oleg234-server.virtnet: executing set_default_debug -1 all [ 1859.910763] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 1860.014841] Lustre: 66140:0:(mgs_llog.c:1348:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 1860.057326] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [ 1860.061290] Lustre: Skipped 1 previous similar message [ 1860.125355] Lustre: lustre-MDT0001: new disk, initializing [ 1860.197405] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 1860.230275] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [ 1860.239760] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 1863.366613] Lustre: DEBUG MARKER: oleg234-server.virtnet: executing set_default_debug -1 all [ 1869.739825] Lustre: DEBUG MARKER: oleg234-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 1872.788052] Lustre: DEBUG MARKER: oleg234-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 1879.324787] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 1879.615919] Lustre: lustre-OST0000: new disk, initializing [ 1879.624164] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 1879.629453] Lustre: 67879:0:(osd_compat.c:1354:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 1879.714639] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 1884.510458] Lustre: DEBUG MARKER: oleg234-server.virtnet: executing set_default_debug -1 all [ 1889.356808] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [ 1889.365039] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 1889.481629] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 1890.784967] Lustre: DEBUG MARKER: oleg234-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 1894.391143] Lustre: DEBUG MARKER: oleg234-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 [ 1894.555366] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [ 1897.618326] Lustre: DEBUG MARKER: oleg234-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid 50 [ 1897.811549] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [ 1900.011775] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 1900.021101] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 1900.024641] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 1900.025027] LustreError: Skipped 1 previous similar message [ 1900.036380] Lustre: Skipped 1 previous similar message [ 1905.122896] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 1905.131463] Lustre: Skipped 2 previous similar messages [ 1905.533809] LustreError: 69069:0:(obd_class.h:479:obd_check_dev()) Device 19 not setup [ 1905.679602] Lustre: server umount lustre-OST0000 complete [ 1910.242529] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 1910.254803] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 1915.360744] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 1915.365656] Lustre: Skipped 2 previous similar messages [ 1915.415541] LustreError: 69270:0:(obd_class.h:479:obd_check_dev()) Device 22 not setup [ 1915.418236] LustreError: 69270:0:(obd_class.h:479:obd_check_dev()) Skipped 3 previous similar messages [ 1915.481984] Lustre: server umount lustre-MDT0000 complete [ 1918.291487] LustreError: 66131:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1770686674 with bad export cookie 8843886153092642641 [ 1918.301988] LustreError: MGC192.168.202.134@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 1918.425358] LustreError: 69471:0:(obd_class.h:479:obd_check_dev()) Device 21 not setup [ 1918.432651] LustreError: 69471:0:(obd_class.h:479:obd_check_dev()) Skipped 5 previous similar messages [ 1918.656882] Lustre: server umount lustre-MDT0001 complete [ 1925.743887] Lustre: DEBUG MARKER: == conf-sanity test 79: format MDT/OST without mgs option (should return errors) ========================================================== 20:24:39 (1770686679) [ 1934.303412] Lustre: DEBUG MARKER: oleg234-server.virtnet: executing load_modules_local [ 1966.873335] Lustre: DEBUG MARKER: oleg234-server.virtnet: executing set_hostid [ 1976.381554] Lustre: DEBUG MARKER: oleg234-server.virtnet: executing load_modules_local [ 2007.645119] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 2007.836678] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 2007.862458] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 2007.917798] Lustre: lustre-MDT0000: new disk, initializing [ 2007.973976] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 2007.984067] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 2011.285498] Lustre: DEBUG MARKER: oleg234-server.virtnet: executing set_default_debug -1 all [ 2020.660304] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 2020.754959] Lustre: 73697:0:(mgs_llog.c:1348:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 2020.792550] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [ 2020.795497] Lustre: Skipped 1 previous similar message [ 2020.872810] Lustre: lustre-MDT0001: new disk, initializing [ 2020.976803] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 2021.035574] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [ 2021.043433] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 2024.292588] Lustre: DEBUG MARKER: oleg234-server.virtnet: executing set_default_debug -1 all [ 2030.422594] Lustre: DEBUG MARKER: oleg234-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 2033.968527] Lustre: DEBUG MARKER: oleg234-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 2038.970722] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 2039.158811] Lustre: lustre-OST0000: new disk, initializing [ 2039.161712] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 2039.172427] Lustre: 75437:0:(osd_compat.c:1354:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 2039.241337] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 2044.002738] Lustre: DEBUG MARKER: oleg234-server.virtnet: executing set_default_debug -1 all [ 2045.007527] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [ 2045.024947] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 2045.135921] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 2050.098075] Lustre: DEBUG MARKER: oleg234-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 2053.752217] Lustre: DEBUG MARKER: oleg234-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 [ 2053.968043] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [ 2057.483217] Lustre: DEBUG MARKER: oleg234-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid 50 [ 2057.739378] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [ 2060.774491] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 2060.778046] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2060.782284] LustreError: Skipped 1 previous similar message [ 2060.782756] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 2060.814365] Lustre: Skipped 2 previous similar messages [ 2065.416855] LustreError: 76627:0:(obd_class.h:479:obd_check_dev()) Device 19 not setup [ 2065.419931] LustreError: 76627:0:(obd_class.h:479:obd_check_dev()) Skipped 7 previous similar messages [ 2065.537852] Lustre: server umount lustre-OST0000 complete [ 2071.010991] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2071.018122] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 2071.028211] Lustre: Skipped 1 previous similar message [ 2071.037323] Lustre: Skipped 2 previous similar messages [ 2074.746694] LustreError: 76829:0:(obd_class.h:479:obd_check_dev()) Device 22 not setup [ 2074.755611] LustreError: 76829:0:(obd_class.h:479:obd_check_dev()) Skipped 3 previous similar messages [ 2074.854169] Lustre: server umount lustre-MDT0000 complete [ 2077.821551] LustreError: 74484:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1770686833 with bad export cookie 8843886153092643523 [ 2077.824412] LustreError: MGC192.168.202.134@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 2077.832682] LustreError: 74484:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) Skipped 6 previous similar messages [ 2078.077656] Lustre: server umount lustre-MDT0001 complete [ 2085.220411] Lustre: DEBUG MARKER: == conf-sanity test 80: mgc import reconnect race ======== 20:27:19 (1770686839) [ 2090.739476] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 2091.118940] LustreError: 77700:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 2091.133642] LustreError: 77700:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 3 previous similar messages [ 2091.175944] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 2094.629711] Lustre: DEBUG MARKER: oleg234-server.virtnet: executing set_default_debug -1 all [ 2096.610056] LustreError: 77700:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 2100.709261] LustreError: 77700:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 2101.832622] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 2105.517870] Lustre: DEBUG MARKER: oleg234-server.virtnet: executing set_default_debug -1 all [ 2110.214919] Lustre: DEBUG MARKER: oleg234-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 2113.426368] Lustre: DEBUG MARKER: oleg234-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 2119.282986] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 2124.204051] Lustre: DEBUG MARKER: oleg234-server.virtnet: executing set_default_debug -1 all [ 2129.815258] Lustre: DEBUG MARKER: oleg234-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 2132.748728] Lustre: 80033:0:(genops.c:1791:obd_export_evict_by_uuid()) MGS: evicting 7b9bd83c-e640-4784-a7b5-dec4fcc32605 at adminstrative request [ 2135.917129] LustreError: MGC192.168.202.134@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 2135.924950] Lustre: *** cfs_fail_loc=906, val=2147483648*** [ 2146.276435] Lustre: MGS: Client 7b9bd83c-e640-4784-a7b5-dec4fcc32605 (at 0@lo) reconnecting [ 2146.283381] Lustre: Evicted from MGS (at 0@lo) after server handle changed from 0x7abbcb996d153e35 to 0x7abbcb996d1540d5 [ 2146.302625] Lustre: MGC192.168.202.134@tcp: Connection restored to 0@lo (at 0@lo) [ 2169.063588] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 2169.192748] Lustre: lustre-OST0001: new disk, initializing [ 2169.196904] Lustre: srv-lustre-OST0001: No data found on store. Initialize space. [ 2169.204603] Lustre: 80425:0:(osd_compat.c:1354:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 2169.273585] Lustre: lustre-OST0001: Imperative Recovery not enabled, recovery window 60-180 [ 2169.276614] Lustre: Skipped 2 previous similar messages [ 2174.867570] Lustre: DEBUG MARKER: oleg234-server.virtnet: executing set_default_debug -1 all [ 2178.123665] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x00000002c0000400-0x0000000300000400]:1:ost [ 2178.133530] Lustre: cli-lustre-OST0001-super: Allocated super-sequence [0x00000002c0000400-0x0000000300000400]:1:ost] [ 2178.307661] Lustre: lustre-OST0001-osc-MDT0000: update sequence from 0x100010000 to 0x2c0000401 [ 2181.931598] Lustre: DEBUG MARKER: oleg234-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0001-osc-[-0-9a-f]*.ost_server_uuid [ 2188.769374] LustreError: lustre-OST0001-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 2188.770312] Lustre: lustre-OST0001-osc-MDT0001: Connection to lustre-OST0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2188.779777] LustreError: Skipped 1 previous similar message [ 2188.782243] Lustre: lustre-OST0001: Not available for connect from 0@lo (stopping) [ 2188.813802] Lustre: Skipped 1 previous similar message [ 2191.053211] LustreError: 81165:0:(obd_class.h:479:obd_check_dev()) Device 25 not setup [ 2191.055996] LustreError: 81165:0:(obd_class.h:479:obd_check_dev()) Skipped 13 previous similar messages [ 2191.127400] Lustre: server umount lustre-OST0001 complete [ 2193.903752] LustreError: 79295:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 2193.918610] LustreError: 79295:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 1 previous similar message [ 2197.472748] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 2197.481801] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2199.010096] LustreError: 80046:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 2199.025636] LustreError: 80046:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 1 previous similar message [ 2209.253328] Lustre: lustre-OST0000 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 2209.256144] LustreError: 79294:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 2209.279989] LustreError: 79294:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 3 previous similar messages [ 2209.306151] LustreError: 81367:0:(obd_class.h:479:obd_check_dev()) Device 21 not setup [ 2209.309935] LustreError: 81367:0:(obd_class.h:479:obd_check_dev()) Skipped 3 previous similar messages [ 2209.378716] Lustre: server umount lustre-OST0000 complete [ 2214.370925] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2214.386379] Lustre: Skipped 2 previous similar messages [ 2218.499640] Lustre: server umount lustre-MDT0000 complete [ 2221.747626] LustreError: 77679:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1770686977 with bad export cookie 8843886153092645077 [ 2221.755195] LustreError: MGC192.168.202.134@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 2221.756428] LustreError: 77679:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 2222.163837] Lustre: server umount lustre-MDT0001 complete [ 2229.610619] Lustre: DEBUG MARKER: == conf-sanity test 81: sparse OST indexing ============== 20:29:43 (1770686983) [ 2230.890589] Lustre: DEBUG MARKER: SKIP: conf-sanity test_81 needs >= 3 OSTs [ 2232.275627] Lustre: DEBUG MARKER: == conf-sanity test 82a: specify OSTs for file (succeed) or directory (succeed) ========================================================== 20:29:46 (1770686986) [ 2233.499940] Lustre: DEBUG MARKER: SKIP: conf-sanity test_82a needs >= 3 OSTs [ 2234.848920] Lustre: DEBUG MARKER: == conf-sanity test 82b: specify OSTs for file with --pool and --ost-list options ========================================================== 20:29:49 (1770686989) [ 2236.118676] Lustre: DEBUG MARKER: SKIP: conf-sanity test_82b needs >= 4 OSTs [ 2237.310881] Lustre: DEBUG MARKER: == conf-sanity test 82c: specify sparse OSTs for setstripe (should not crash) ========================================================== 20:29:51 (1770686991) [ 2238.718489] Lustre: DEBUG MARKER: SKIP: conf-sanity test_82c needs >= 4 OSTs [ 2240.155511] Lustre: DEBUG MARKER: == conf-sanity test 83: ENOSPACE on OST doesn't cause message VFS: Busy inodes after unmount ... ========================================================== 20:29:54 (1770686994) [ 2241.444979] Lustre: DEBUG MARKER: mount the OST /dev/mapper/ost1_flakey as a ldiskfs filesystem [ 2246.302839] loop: module loaded [ 2246.329150] loop0: detected capacity change from 0 to 4294967296 [ 2246.427638] LDISKFS-fs (loop0): mounted filesystem with ordered data mode. Opts: (null) [ 2247.805087] Lustre: DEBUG MARKER: run llverfs in partial mode on the OST ldiskfs /mnt/lustre-ost1 [ 2250.383165] Lustre: DEBUG MARKER: oleg234-server.virtnet: executing run_llverfs /mnt/lustre-ost1 -vpl no [ 2253.355954] Lustre: DEBUG MARKER: unmount the OST /dev/mapper/ost1_flakey [ 2275.000355] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 2275.029380] LustreError: 84414:0:(obd_config.c:783:class_setup()) setup lustre-OST0000-osd failed (-28) [ 2275.032265] LustreError: 84414:0:(obd_mount.c:193:lustre_start_simple()) lustre-OST0000-osd setup error -28 [ 2275.034990] LustreError: 84414:0:(tgt_mount.c:2451:server_fill_super()) Unable to start osd on /dev/mapper/ost1_flakey: -28 [ 2275.038476] LustreError: 84414:0:(super25.c:199:lustre_fill_super()) llite: Unable to mount : rc = -28 [ 2290.571512] Lustre: DEBUG MARKER: oleg234-server.virtnet: executing set_hostid [ 2298.873251] Lustre: DEBUG MARKER: oleg234-server.virtnet: executing load_modules_local [ 2328.655724] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 2328.834089] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 2328.864806] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 2328.908781] Lustre: lustre-MDT0000: new disk, initializing [ 2328.960805] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 2328.978442] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 2331.572815] Lustre: DEBUG MARKER: oleg234-server.virtnet: executing set_default_debug -1 all [ 2339.203356] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 2339.305257] Lustre: 87247:0:(mgs_llog.c:1348:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 2339.550440] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 2342.473111] Lustre: DEBUG MARKER: oleg234-server.virtnet: executing set_default_debug -1 all [ 2348.282392] Lustre: DEBUG MARKER: oleg234-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 2351.205076] Lustre: DEBUG MARKER: oleg234-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 2355.961432] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 2356.165442] Lustre: 88985:0:(osd_compat.c:1354:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 2360.185852] Lustre: DEBUG MARKER: oleg234-server.virtnet: executing set_default_debug -1 all [ 2361.900724] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 2361.970057] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 2365.916882] Lustre: DEBUG MARKER: oleg234-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 2369.006540] Lustre: DEBUG MARKER: oleg234-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 [ 2369.151851] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [ 2372.023043] Lustre: DEBUG MARKER: oleg234-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid 50 [ 2372.180400] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [ 2377.184249] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 2377.185903] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2377.190813] LustreError: Skipped 2 previous similar messages [ 2377.204681] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 2377.211812] Lustre: Skipped 12 previous similar messages [ 2379.574776] LustreError: 90175:0:(obd_class.h:479:obd_check_dev()) Device 19 not setup [ 2379.577727] LustreError: 90175:0:(obd_class.h:479:obd_check_dev()) Skipped 21 previous similar messages [ 2379.653742] Lustre: server umount lustre-OST0000 complete [ 2385.888716] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 2388.586793] Lustre: server umount lustre-MDT0000 complete [ 2391.552252] LustreError: 87238:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1770687147 with bad export cookie 8843886153092645672 [ 2391.554415] LustreError: MGC192.168.202.134@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 2391.561329] LustreError: 87238:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 2391.748144] Lustre: server umount lustre-MDT0001 complete [ 2397.842360] Lustre: DEBUG MARKER: == conf-sanity test 84: check recovery_hard_time ========= 20:32:32 (1770687152) [ 2403.040389] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 2403.551801] LustreError: 91249:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 2403.568159] LustreError: 91249:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 2 previous similar messages [ 2407.709754] Lustre: DEBUG MARKER: oleg234-server.virtnet: executing set_default_debug -1 all [ 2414.450728] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 2417.363829] Lustre: DEBUG MARKER: oleg234-server.virtnet: executing set_default_debug -1 all [ 2421.547257] Lustre: DEBUG MARKER: oleg234-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 2424.626450] Lustre: DEBUG MARKER: oleg234-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 2429.740193] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 2433.771264] Lustre: DEBUG MARKER: oleg234-server.virtnet: executing set_default_debug -1 all [ 2437.703749] Lustre: DEBUG MARKER: oleg234-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 2440.798752] Lustre: DEBUG MARKER: oleg234-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 [ 2449.368288] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 8 sec [ 2452.025722] Lustre: DEBUG MARKER: oleg234-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid 50 [ 2452.136875] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [ 2456.386148] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 2456.495327] Lustre: lustre-OST0001: new disk, initializing [ 2456.498497] Lustre: Skipped 2 previous similar messages [ 2456.502284] Lustre: srv-lustre-OST0001: No data found on store. Initialize space. [ 2456.504971] Lustre: Skipped 3 previous similar messages [ 2456.508416] Lustre: 94302:0:(osd_compat.c:1354:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 2460.205643] Lustre: DEBUG MARKER: oleg234-server.virtnet: executing set_default_debug -1 all [ 2465.368405] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x00000002c0000400-0x0000000300000400]:1:ost [ 2465.376108] Lustre: Skipped 2 previous similar messages [ 2465.384517] Lustre: cli-lustre-OST0001-super: Allocated super-sequence [0x00000002c0000400-0x0000000300000400]:1:ost] [ 2465.487611] Lustre: lustre-OST0001-osc-MDT0000: update sequence from 0x100010000 to 0x2c0000401 [ 2465.884530] Lustre: DEBUG MARKER: oleg234-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0001-osc-[-0-9a-f]*.ost_server_uuid [ 2468.908566] Lustre: DEBUG MARKER: oleg234-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0001-osc-MDT0000.ost_server_uuid 50 [ 2469.027230] Lustre: DEBUG MARKER: os[cp].lustre-OST0001-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [ 2471.961219] Lustre: DEBUG MARKER: oleg234-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0001-osc-MDT0001.ost_server_uuid 50 [ 2472.101382] Lustre: DEBUG MARKER: os[cp].lustre-OST0001-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [ 2485.841361] Lustre: DEBUG MARKER: mds1 REPLAY BARRIER on lustre-MDT0000 [ 2505.221912] Lustre: Failing over lustre-MDT0000 [ 2505.416898] LustreError: 96204:0:(obd_class.h:479:obd_check_dev()) Device 28 not setup [ 2505.423722] LustreError: 96204:0:(obd_class.h:479:obd_check_dev()) Skipped 17 previous similar messages [ 2505.472041] Lustre: server umount lustre-MDT0000 complete [ 2506.729845] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 2506.735743] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2506.747653] Lustre: Skipped 3 previous similar messages [ 2506.758165] LustreError: 91248:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 2506.776168] LustreError: 91248:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 2 previous similar messages [ 2526.175335] Lustre: 64365:0:(client.c:2478:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1770687265/real 1770687265] req@000000002da7a10f x1856699442112768/t0(0) o400->MGC192.168.202.134@tcp@0@lo:26/25 lens 224/224 e 0 to 1 dl 1770687281 ref 1 fl Rpc:XNQr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 [ 2526.201206] LustreError: MGC192.168.202.134@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 2527.272475] LDISKFS-fs (dm-0): 4 truncates cleaned up [ 2527.274908] LDISKFS-fs (dm-0): recovery complete [ 2527.286589] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 2535.395502] LustreError: 64364:0:(client.c:1390:ptlrpc_import_delay_req()) @@@ invalidate in flight req@00000000b2c765af x1856699442121216/t0(0) o250->MGC192.168.202.134@tcp@0@lo:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl Rpc:NQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 [ 2535.604941] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-60 [ 2535.609097] Lustre: Skipped 6 previous similar messages [ 2535.647420] Lustre: lustre-MDT0000: in recovery but waiting for the first client to connect [ 2536.370129] Lustre: lustre-MDT0000: Will be in recovery for at least 1:00, or until 3 clients reconnect [ 2536.372159] Lustre: 91243:0:(ldlm_lib.c:2068:extend_recovery_timer()) lustre-MDT0000: extended recovery timer reached hard limit: 60, extend: 0 [ 2538.496884] Lustre: DEBUG MARKER: oleg234-server.virtnet: executing set_default_debug -1 all [ 2541.033090] Lustre: 91248:0:(ldlm_lib.c:2068:extend_recovery_timer()) lustre-MDT0000: extended recovery timer reached hard limit: 60, extend: 0 [ 2541.045804] Lustre: 91248:0:(ldlm_lib.c:2068:extend_recovery_timer()) Skipped 1 previous similar message [ 2541.067372] Lustre: lustre-MDT0000-lwp-MDT0001: Connection restored to 0@lo (at 0@lo) [ 2541.128108] LustreError: 96950:0:(ldlm_lib.c:2681:replay_request_or_update()) cfs_fail_timeout id 709 sleeping for 300ms [ 2541.450678] LustreError: 96950:0:(ldlm_lib.c:2681:replay_request_or_update()) cfs_fail_timeout id 709 awake [ 2541.784689] LustreError: 96950:0:(ldlm_lib.c:2681:replay_request_or_update()) cfs_fail_timeout id 709 sleeping for 300ms [ 2541.792106] LustreError: 96950:0:(ldlm_lib.c:2681:replay_request_or_update()) Skipped 1 previous similar message [ 2542.120709] LustreError: 96950:0:(ldlm_lib.c:2681:replay_request_or_update()) cfs_fail_timeout id 709 awake [ 2542.129602] LustreError: 96950:0:(ldlm_lib.c:2681:replay_request_or_update()) Skipped 1 previous similar message [ 2542.144487] Lustre: 96950:0:(ldlm_lib.c:2068:extend_recovery_timer()) lustre-MDT0000: extended recovery timer reached hard limit: 60, extend: 1 [ 2542.157800] Lustre: 96950:0:(ldlm_lib.c:2068:extend_recovery_timer()) Skipped 7 previous similar messages [ 2542.810000] LustreError: 96950:0:(ldlm_lib.c:2681:replay_request_or_update()) cfs_fail_timeout id 709 sleeping for 300ms [ 2542.821979] LustreError: 96950:0:(ldlm_lib.c:2681:replay_request_or_update()) Skipped 2 previous similar messages [ 2543.143139] LustreError: 96950:0:(ldlm_lib.c:2681:replay_request_or_update()) cfs_fail_timeout id 709 awake [ 2543.147470] LustreError: 96950:0:(ldlm_lib.c:2681:replay_request_or_update()) Skipped 2 previous similar messages [ 2544.409435] Lustre: 96950:0:(ldlm_lib.c:2068:extend_recovery_timer()) lustre-MDT0000: extended recovery timer reached hard limit: 60, extend: 1 [ 2544.421043] Lustre: 96950:0:(ldlm_lib.c:2068:extend_recovery_timer()) Skipped 6 previous similar messages [ 2545.061221] LustreError: 96950:0:(ldlm_lib.c:2681:replay_request_or_update()) cfs_fail_timeout id 709 sleeping for 300ms [ 2545.072425] LustreError: 96950:0:(ldlm_lib.c:2681:replay_request_or_update()) Skipped 6 previous similar messages [ 2545.394641] LustreError: 96950:0:(ldlm_lib.c:2681:replay_request_or_update()) cfs_fail_timeout id 709 awake [ 2545.397951] LustreError: 96950:0:(ldlm_lib.c:2681:replay_request_or_update()) Skipped 6 previous similar messages [ 2548.562666] Lustre: 96950:0:(ldlm_lib.c:2068:extend_recovery_timer()) lustre-MDT0000: extended recovery timer reached hard limit: 60, extend: 1 [ 2548.582400] Lustre: 96950:0:(ldlm_lib.c:2068:extend_recovery_timer()) Skipped 12 previous similar messages [ 2549.223948] LustreError: 96950:0:(ldlm_lib.c:2681:replay_request_or_update()) cfs_fail_timeout id 709 sleeping for 300ms [ 2549.229724] LustreError: 96950:0:(ldlm_lib.c:2681:replay_request_or_update()) Skipped 12 previous similar messages [ 2549.543101] LustreError: 96950:0:(ldlm_lib.c:2681:replay_request_or_update()) cfs_fail_timeout id 709 awake [ 2549.546923] LustreError: 96950:0:(ldlm_lib.c:2681:replay_request_or_update()) Skipped 12 previous similar messages [ 2556.808211] Lustre: 96950:0:(ldlm_lib.c:2068:extend_recovery_timer()) lustre-MDT0000: extended recovery timer reached hard limit: 60, extend: 1 [ 2556.817670] Lustre: 96950:0:(ldlm_lib.c:2068:extend_recovery_timer()) Skipped 25 previous similar messages [ 2557.464154] LustreError: 96950:0:(ldlm_lib.c:2681:replay_request_or_update()) cfs_fail_timeout id 709 sleeping for 300ms [ 2557.468242] LustreError: 96950:0:(ldlm_lib.c:2681:replay_request_or_update()) Skipped 25 previous similar messages [ 2557.791211] LustreError: 96950:0:(ldlm_lib.c:2681:replay_request_or_update()) cfs_fail_timeout id 709 awake [ 2557.796485] LustreError: 96950:0:(ldlm_lib.c:2681:replay_request_or_update()) Skipped 25 previous similar messages [ 2572.898793] Lustre: 96950:0:(ldlm_lib.c:2068:extend_recovery_timer()) lustre-MDT0000: extended recovery timer reached hard limit: 60, extend: 1 [ 2572.908346] Lustre: 96950:0:(ldlm_lib.c:2068:extend_recovery_timer()) Skipped 50 previous similar messages [ 2573.537882] LustreError: 96950:0:(ldlm_lib.c:2681:replay_request_or_update()) cfs_fail_timeout id 709 sleeping for 300ms [ 2573.544821] LustreError: 96950:0:(ldlm_lib.c:2681:replay_request_or_update()) Skipped 50 previous similar messages [ 2573.866429] LustreError: 96950:0:(ldlm_lib.c:2681:replay_request_or_update()) cfs_fail_timeout id 709 awake [ 2573.872517] LustreError: 96950:0:(ldlm_lib.c:2681:replay_request_or_update()) Skipped 50 previous similar messages [ 2596.191826] Lustre: 96950:0:(ldlm_lib.c:2377:target_recovery_overseer()) lustre-MDT0000 recovery is aborted by hard timeout [ 2596.201205] Lustre: 96950:0:(ldlm_lib.c:2387:target_recovery_overseer()) recovery is aborted, evict exports in recovery [ 2596.205597] Lustre: 96950:0:(genops.c:1618:class_disconnect_stale_exports()) lustre-MDT0000: disconnect stale client 01060192-b847-4207-83fe-9229fd966167@192.168.202.34@tcp [ 2596.212951] Lustre: lustre-MDT0000: disconnecting 1 stale clients [ 2596.216629] Lustre: 96950:0:(ldlm_lib.c:1897:abort_req_replay_queue()) @@@ aborted: req@000000007ae928f2 x1856699426211072/t0(8589934771) o101->01060192-b847-4207-83fe-9229fd966167@192.168.202.34@tcp:207/0 lens 592/0 e 0 to 0 dl 1770687362 ref 1 fl Complete:/604/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:0 [ 2596.232951] Lustre: lustre-MDT0000-osd: cancel update llog [0x200000400:0x1:0x0] [ 2596.241928] Lustre: lustre-MDT0000: Denying connection for new client 01060192-b847-4207-83fe-9229fd966167 (at 192.168.202.34@tcp), waiting for 3 known clients (2 recovered, 0 in progress, and 1 evicted) to recover in 0:00 [ 2596.262339] Lustre: lustre-MDT0001-osp-MDT0000: cancel update llog [0x240000401:0x1:0x0] [ 2596.287238] Lustre: lustre-MDT0000-osp-MDT0001: Connection restored to 0@lo (at 0@lo) [ 2596.290482] Lustre: Skipped 2 previous similar messages [ 2596.293672] Lustre: lustre-MDT0000: Recovery over after 1:00, of 3 clients 2 recovered and 1 was evicted. [ 2596.328223] Lustre: lustre-OST0001: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x2c0000401:46 to 0x2c0000401:545) [ 2596.330593] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:77 to 0x280000401:577) [ 2609.121649] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 2609.131994] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 2609.134871] Lustre: Skipped 3 previous similar messages [ 2613.810773] Lustre: server umount lustre-OST0000 complete [ 2617.829408] LustreError: 92840:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-OST0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 2617.842162] LustreError: 92840:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 41 previous similar messages [ 2622.372501] Lustre: server umount lustre-OST0001 complete [ 2631.444506] Lustre: server umount lustre-MDT0000 complete [ 2634.699496] LustreError: 91230:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1770687390 with bad export cookie 8843886153092696884 [ 2634.712728] LustreError: MGC192.168.202.134@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 2634.842651] LustreError: 98215:0:(obd_class.h:479:obd_check_dev()) Device 27 not setup [ 2634.848730] LustreError: 98215:0:(obd_class.h:479:obd_check_dev()) Skipped 23 previous similar messages [ 2634.990878] Lustre: server umount lustre-MDT0001 complete [ 2641.126313] Lustre: DEBUG MARKER: == conf-sanity test 85: osd_ost init: fail ea_fid_set ==== 20:36:35 (1770687395) [ 2646.073911] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 2646.079160] Lustre: *** cfs_fail_loc=197, val=0*** [ 2661.601858] LustreError: Failed to get MGS log lustre-sptlrpc and no local copy. [ 2666.722539] LustreError: 98907:0:(mgc_request.c:233:do_config_log_add()) MGC192.168.202.134@tcp: failed processing log, type 4: rc = -110 [ 2671.852146] Lustre: *** cfs_fail_loc=197, val=0*** [ 2671.856962] Lustre: Skipped 65 previous similar messages [ 2695.747048] Lustre: DEBUG MARKER: oleg234-server.virtnet: executing set_default_debug -1 all [ 2699.023083] Lustre: DEBUG MARKER: oleg234-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 2700.238907] Lustre: server umount lustre-OST0000 complete [ 2706.005988] Lustre: DEBUG MARKER: == conf-sanity test 86: Replacing mkfs.lustre -G option == 20:37:40 (1770687460) [ 2723.313310] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 2725.981365] Lustre: DEBUG MARKER: oleg234-server.virtnet: executing set_default_debug -1 all [ 2730.844090] Lustre: DEBUG MARKER: oleg234-client.virtnet: executing wait_import_state_mount (FULL|IDLE) mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 2733.585218] LustreError: 100441:0:(osp_object.c:617:osp_attr_get()) lustre-MDT0001-osp-MDT0000: osp_attr_get update error [0x200000009:0x1:0x0]: rc = -5 [ 2733.598399] LustreError: 100441:0:(lod_sub_object.c:917:lod_sub_prep_llog()) lustre-MDT0000-mdtlov: can't get id from catalogs: rc = -5 [ 2733.608834] LustreError: 100441:0:(lod_dev.c:510:lod_sub_recovery_thread()) lustre-MDT0001-osp-MDT0000: get update log duration 10, retries 0, failed: rc = -5 [ 2733.819922] Lustre: server umount lustre-MDT0000 complete [ 2753.618686] Lustre: DEBUG MARKER: oleg234-server.virtnet: executing set_hostid [ 2759.179950] Lustre: DEBUG MARKER: oleg234-server.virtnet: executing load_modules_local [ 2778.464129] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 2778.579647] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 2778.597467] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 2778.636491] Lustre: lustre-MDT0000: new disk, initializing [ 2778.690654] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 2780.738927] Lustre: DEBUG MARKER: oleg234-server.virtnet: executing set_default_debug -1 all [ 2787.569718] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 2787.648396] Lustre: 104461:0:(mgs_llog.c:1348:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 2787.812374] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 2790.356709] Lustre: DEBUG MARKER: oleg234-server.virtnet: executing set_default_debug -1 all [ 2795.365685] Lustre: DEBUG MARKER: oleg234-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 2797.214842] Lustre: DEBUG MARKER: oleg234-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 2800.432112] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 2800.584179] Lustre: 106202:0:(osd_compat.c:1354:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 2800.624883] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 2800.628484] Lustre: Skipped 4 previous similar messages [ 2803.847234] Lustre: DEBUG MARKER: oleg234-server.virtnet: executing set_default_debug -1 all [ 2807.892354] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 2808.613435] Lustre: DEBUG MARKER: oleg234-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 2811.075925] Lustre: DEBUG MARKER: oleg234-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 [ 2811.201342] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [ 2813.490644] Lustre: DEBUG MARKER: oleg234-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid 50 [ 2813.596853] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [ 2818.022107] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2818.022428] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 2818.031125] Lustre: Skipped 9 previous similar messages [ 2818.039130] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 2818.040525] LustreError: Skipped 4 previous similar messages [ 2818.052939] Lustre: Skipped 8 previous similar messages [ 2829.279311] Lustre: lustre-OST0000 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 2829.366091] Lustre: server umount lustre-OST0000 complete [ 2837.799352] Lustre: server umount lustre-MDT0000 complete [ 2838.499574] LustreError: 105519:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 2838.508801] LustreError: 105519:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 5 previous similar messages [ 2840.046071] LustreError: 104454:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1770687595 with bad export cookie 8843886153092707461 [ 2840.053613] LustreError: MGC192.168.202.134@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 2840.058625] LustreError: 104454:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) Skipped 8 previous similar messages [ 2840.202729] Lustre: server umount lustre-MDT0001 complete [ 2845.285349] Lustre: DEBUG MARKER: == conf-sanity test 87: check if MDT inode can hold EAs with N stripes properly ========================================================== 20:39:59 (1770687599) [ 2850.497600] Lustre: DEBUG MARKER: oleg234-server.virtnet: executing unload_modules_local [ 2852.043362] Key type lgssc unregistered [ 2852.216036] LNet: 108521:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 2852.219471] LNetError: 108521:0:(acceptor.c:264:lnet_acceptor_remove_socket()) Interface ens2 not found [ 2852.233488] LNet: Removed LNI 192.168.202.134@tcp [ 2852.675967] Key type .llcrypt unregistered [ 2852.677737] Key type ._llcrypt unregistered [ 2863.362872] Key type ._llcrypt registered [ 2863.364384] Key type .llcrypt registered [ 2863.409720] Lustre: DEBUG MARKER: oleg234-server.virtnet: executing set_hostid [ 2873.993250] Lustre: DEBUG MARKER: oleg234-server.virtnet: executing load_modules_local [ 2874.513703] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 2874.526463] alg: No test for adler32 (adler32-zlib) [ 2875.462258] Lustre: Lustre: Build Version: 2.17.50_65_g69bb343 [ 2875.607109] LNet: Added LNI 192.168.202.134@tcp [8/256/0/180] [ 2877.232211] Key type lgssc registered [ 2877.887426] Lustre: Echo OBD driver; http://www.lustre.org/ [ 2902.213986] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 2902.226673] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 2903.373914] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 2903.397353] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 2903.449640] Lustre: lustre-MDT0000: new disk, initializing [ 2903.496214] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 2903.515553] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 2905.954695] Lustre: DEBUG MARKER: oleg234-server.virtnet: executing set_default_debug -1 all [ 2912.318291] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 2912.453388] Lustre: lustre-OST0000: new disk, initializing [ 2912.455718] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 2912.462949] Lustre: Skipped 1 previous similar message [ 2912.467439] Lustre: 112616:0:(osd_compat.c:1354:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 2912.504670] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 2915.476795] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:0:ost [ 2915.482167] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:0:ost] [ 2915.500266] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x240000400 [ 2915.810314] Lustre: DEBUG MARKER: oleg234-server.virtnet: executing set_default_debug -1 all [ 2922.117570] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 2922.226970] Lustre: lustre-OST0001: new disk, initializing [ 2922.229702] Lustre: srv-lustre-OST0001: No data found on store. Initialize space. [ 2922.237777] Lustre: 113463:0:(osd_compat.c:1354:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 2922.277776] Lustre: lustre-OST0001: Imperative Recovery not enabled, recovery window 60-180 [ 2925.647819] Lustre: DEBUG MARKER: oleg234-server.virtnet: executing set_default_debug -1 all [ 2928.203243] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:1:ost [ 2928.207979] Lustre: cli-lustre-OST0001-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:1:ost] [ 2928.227073] Lustre: lustre-OST0001-osc-MDT0000: update sequence from 0x100010000 to 0x280000400 [ 2933.218427] Lustre: lustre-MDT0000-lwp-OST0000: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2933.222280] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 2933.223957] Lustre: Skipped 1 previous similar message [ 2937.031718] LustreError: 114056:0:(obd_class.h:479:obd_check_dev()) Device 17 not setup [ 2937.105718] Lustre: server umount lustre-MDT0000 complete [ 2940.560633] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: (null) [ 2943.091423] LustreError: 111821:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1770687698 with bad export cookie 17783291419945858846 [ 2943.095048] LustreError: MGC192.168.202.134@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 2943.096928] LustreError: 111821:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) Skipped 1 previous similar message [ 2943.120172] LustreError: 114650:0:(obd_class.h:479:obd_check_dev()) Device 12 not setup [ 2943.123242] LustreError: 114650:0:(obd_class.h:479:obd_check_dev()) Skipped 5 previous similar messages [ 2943.142659] Lustre: server umount lustre-OST0000 complete [ 2944.988661] LustreError: 114850:0:(obd_class.h:479:obd_check_dev()) Device 2 not setup [ 2944.993431] LustreError: 114850:0:(obd_class.h:479:obd_check_dev()) Skipped 1 previous similar message [ 2945.054988] Lustre: server umount lustre-OST0001 complete [ 2949.210806] Lustre: DEBUG MARKER: == conf-sanity test 88: check the default mount options can be overridden ========================================================== 20:41:44 (1770687704) [ 2953.127504] Lustre: DEBUG MARKER: oleg234-server.virtnet: executing load_modules_local [ 2963.617730] Lustre: DEBUG MARKER: == conf-sanity test 89: check tunefs --param and --erase-param{s} options ========================================================== 20:41:58 (1770687718) [ 2982.723137] Lustre: DEBUG MARKER: oleg234-server.virtnet: executing set_hostid [ 2987.859040] Lustre: DEBUG MARKER: oleg234-server.virtnet: executing load_modules_local [ 3005.006955] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 3005.144147] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 3005.170590] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 3005.228085] Lustre: lustre-MDT0000: new disk, initializing [ 3005.274050] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 3005.287886] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 3007.054272] Lustre: DEBUG MARKER: oleg234-server.virtnet: executing set_default_debug -1 all [ 3012.308649] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 3012.382515] Lustre: 120135:0:(mgs_llog.c:1348:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 3012.411920] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [ 3012.415560] Lustre: Skipped 1 previous similar message [ 3012.455824] Lustre: lustre-MDT0001: new disk, initializing [ 3012.501094] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 3012.518777] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [ 3012.524810] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 3014.346332] Lustre: DEBUG MARKER: oleg234-server.virtnet: executing set_default_debug -1 all [ 3018.260570] Lustre: DEBUG MARKER: oleg234-client.virtnet: executing wait_import_state_mount FULL failover.node=192.0.2.254@tcp0,mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 3020.146903] Lustre: DEBUG MARKER: oleg234-client.virtnet: executing wait_import_state_mount FULL failover.node=192.0.2.254@tcp0,mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 3023.851906] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 3023.982103] Lustre: lustre-OST0000: new disk, initializing [ 3023.984318] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 3023.989290] Lustre: 121873:0:(osd_compat.c:1354:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 3024.030277] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 3025.736719] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [ 3025.741633] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 3025.801991] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 3027.094076] Lustre: DEBUG MARKER: oleg234-server.virtnet: executing set_default_debug -1 all [ 3031.394171] Lustre: DEBUG MARKER: oleg234-client.virtnet: executing wait_import_state_mount (FULL|IDLE) failover.node=192.0.2.254@tcp0,osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 3033.458280] Lustre: DEBUG MARKER: oleg234-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 [ 3033.549876] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [ 3035.475904] Lustre: DEBUG MARKER: oleg234-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid 50 [ 3035.582737] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [ 3041.249941] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 3041.252971] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 3041.259503] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 3041.261861] Lustre: Skipped 2 previous similar messages [ 3042.483132] LustreError: 123063:0:(obd_class.h:479:obd_check_dev()) Device 19 not setup [ 3042.485567] LustreError: 123063:0:(obd_class.h:479:obd_check_dev()) Skipped 3 previous similar messages [ 3042.531456] Lustre: server umount lustre-OST0000 complete [ 3046.369345] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 3046.369889] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 3046.374810] Lustre: Skipped 2 previous similar messages [ 3050.241116] LustreError: 123264:0:(obd_class.h:479:obd_check_dev()) Device 22 not setup [ 3050.243589] LustreError: 123264:0:(obd_class.h:479:obd_check_dev()) Skipped 3 previous similar messages [ 3050.278142] Lustre: server umount lustre-MDT0000 complete [ 3051.890927] LustreError: 123265:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1770687807 with bad export cookie 17783291419945860064 [ 3051.893388] LustreError: MGC192.168.202.134@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 3051.895268] LustreError: 123265:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 3052.026924] Lustre: server umount lustre-MDT0001 complete [ 3056.293244] Lustre: DEBUG MARKER: == conf-sanity test 90a: check max_mod_rpcs_in_flight is enforced ========================================================== 20:43:31 (1770687811) [ 3059.477527] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 3059.691473] LustreError: 124136:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 3059.706357] LustreError: 124136:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 3 previous similar messages [ 3059.727805] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 3061.646435] Lustre: DEBUG MARKER: oleg234-server.virtnet: executing set_default_debug -1 all [ 3064.803450] LustreError: 124136:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 3066.272158] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 3068.668820] Lustre: DEBUG MARKER: oleg234-server.virtnet: executing set_default_debug -1 all [ 3071.894538] Lustre: DEBUG MARKER: oleg234-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 3073.860915] Lustre: DEBUG MARKER: oleg234-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 3077.407992] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 3080.284629] Lustre: DEBUG MARKER: oleg234-server.virtnet: executing set_default_debug -1 all [ 3082.873470] Lustre: DEBUG MARKER: oleg234-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 3094.531596] Lustre: *** cfs_fail_loc=159, val=0*** [ 3110.845697] Lustre: lustre-MDT0000: Client fa84083b-87de-44ee-81f9-b6e447dc92bc (at 192.168.202.34@tcp) reconnecting [ 3111.637944] Lustre: *** cfs_fail_loc=159, val=0*** [ 3111.642713] Lustre: Skipped 5 previous similar messages [ 3128.257174] Lustre: lustre-MDT0000: Client fa84083b-87de-44ee-81f9-b6e447dc92bc (at 192.168.202.34@tcp) reconnecting [ 3146.213769] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 3146.217594] LustreError: Skipped 1 previous similar message [ 3146.220871] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 3146.227771] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 3146.230389] Lustre: Skipped 1 previous similar message [ 3148.257609] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 3149.681088] LustreError: 126640:0:(obd_class.h:479:obd_check_dev()) Device 21 not setup [ 3149.684832] LustreError: 126640:0:(obd_class.h:479:obd_check_dev()) Skipped 13 previous similar messages [ 3149.759983] Lustre: server umount lustre-OST0000 complete [ 3153.377675] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 3153.377791] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 3153.379408] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 3153.379416] Lustre: Skipped 2 previous similar messages [ 3153.383288] Lustre: Skipped 2 previous similar messages [ 3157.852687] Lustre: server umount lustre-MDT0000 complete [ 3158.496167] LustreError: 124132:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 3158.502601] LustreError: 124132:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 1 previous similar message [ 3159.369603] LustreError: 126537:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1770687915 with bad export cookie 17783291419945860953 [ 3159.372740] LustreError: MGC192.168.202.134@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 3159.374882] LustreError: 126537:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 3159.536942] Lustre: server umount lustre-MDT0001 complete [ 3163.896025] Lustre: DEBUG MARKER: oleg234-server.virtnet: executing unload_modules_local [ 3165.128937] Key type lgssc unregistered [ 3165.265782] LNet: 127573:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 3165.270352] LNetError: 127573:0:(acceptor.c:264:lnet_acceptor_remove_socket()) Interface ens2 not found [ 3165.283338] LNet: Removed LNI 192.168.202.134@tcp [ 3165.633369] Key type .llcrypt unregistered [ 3165.635291] Key type ._llcrypt unregistered [ 3176.083062] Key type ._llcrypt registered [ 3176.084630] Key type .llcrypt registered [ 3176.127144] Lustre: DEBUG MARKER: oleg234-server.virtnet: executing load_modules_local [ 3176.450368] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 3176.457630] alg: No test for adler32 (adler32-zlib) [ 3177.324207] Lustre: Lustre: Build Version: 2.17.50_65_g69bb343 [ 3177.415452] LNet: Added LNI 192.168.202.134@tcp [8/256/0/180] [ 3178.999191] Key type lgssc registered [ 3179.422221] Lustre: Echo OBD driver; http://www.lustre.org/ [ 3182.538395] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 3182.546217] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 3183.742542] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 3185.226253] Lustre: DEBUG MARKER: oleg234-server.virtnet: executing set_default_debug -1 all [ 3188.822070] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 3188.990024] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 3190.543772] Lustre: DEBUG MARKER: oleg234-server.virtnet: executing set_default_debug -1 all [ 3192.703172] Lustre: DEBUG MARKER: oleg234-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 3194.159415] Lustre: DEBUG MARKER: oleg234-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 3197.000737] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 3197.161184] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 3199.479404] Lustre: DEBUG MARKER: oleg234-server.virtnet: executing set_default_debug -1 all [ 3201.735920] Lustre: DEBUG MARKER: oleg234-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 3208.428889] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:42 to 0x280000401:65) [ 3217.414171] Lustre: *** cfs_fail_loc=159, val=0*** [ 3232.702431] Lustre: lustre-MDT0000: Client 92ac87b5-9142-4b22-b47a-b74d9a800edf (at 192.168.202.34@tcp) reconnecting [ 3233.935793] Lustre: *** cfs_fail_loc=159, val=0*** [ 3250.109802] Lustre: lustre-MDT0001: Client 92ac87b5-9142-4b22-b47a-b74d9a800edf (at 192.168.202.34@tcp) reconnecting [ 3250.875504] Lustre: *** cfs_fail_loc=159, val=0*** [ 3250.877413] Lustre: Skipped 3 previous similar messages [ 3266.496841] Lustre: lustre-MDT0001: Client 92ac87b5-9142-4b22-b47a-b74d9a800edf (at 192.168.202.34@tcp) reconnecting [ 3267.848098] Lustre: 131667:0:(mdt_handler.c:64:mdt_max_mod_rpcs_per_client_set()) max_mod_rpcs_per_client is deprecated, set mdt.*.max_mod_rpcs_in_flight parameter instead [ 3268.881300] Lustre: *** cfs_fail_loc=159, val=0*** [ 3268.887684] Lustre: Skipped 4 previous similar messages [ 3285.436961] Lustre: lustre-MDT0000: Client e3a3c1f9-7da5-4f4b-b8a6-4840dd8d8dda (at 192.168.202.34@tcp) reconnecting [ 3286.270405] Lustre: *** cfs_fail_loc=159, val=0*** [ 3286.272249] Lustre: Skipped 14 previous similar messages [ 3301.826101] Lustre: lustre-MDT0000: Client e3a3c1f9-7da5-4f4b-b8a6-4840dd8d8dda (at 192.168.202.34@tcp) reconnecting [ 3302.500341] Lustre: 131958:0:(mdt_handler.c:64:mdt_max_mod_rpcs_per_client_set()) max_mod_rpcs_per_client is deprecated, set mdt.*.max_mod_rpcs_in_flight parameter instead [ 3305.956072] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 3305.956280] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 3305.957980] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 3305.961381] Lustre: Skipped 1 previous similar message [ 3310.257255] LustreError: 132056:0:(obd_class.h:479:obd_check_dev()) Device 21 not setup [ 3310.321666] Lustre: server umount lustre-OST0000 complete [ 3312.096921] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 3312.100139] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 3312.105332] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 3312.107517] Lustre: Skipped 1 previous similar message [ 3316.192220] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 3316.194260] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 3318.151901] LustreError: 132258:0:(obd_class.h:479:obd_check_dev()) Device 9 not setup [ 3318.153875] LustreError: 132258:0:(obd_class.h:479:obd_check_dev()) Skipped 3 previous similar messages [ 3318.186305] Lustre: server umount lustre-MDT0000 complete [ 3319.885550] LustreError: 129663:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1770688075 with bad export cookie 14077511914870722336 [ 3319.890508] LustreError: MGC192.168.202.134@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 3319.893315] LustreError: 129663:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 3319.990955] LustreError: 132459:0:(obd_class.h:479:obd_check_dev()) Device 16 not setup [ 3319.994639] LustreError: 132459:0:(obd_class.h:479:obd_check_dev()) Skipped 5 previous similar messages [ 3320.082567] Lustre: server umount lustre-MDT0001 complete [ 3324.568123] Lustre: DEBUG MARKER: oleg234-server.virtnet: executing unload_modules_local [ 3325.778332] Key type lgssc unregistered [ 3325.915377] LNet: 132990:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 3325.919628] LNetError: 132990:0:(acceptor.c:264:lnet_acceptor_remove_socket()) Interface ens2 not found [ 3325.927370] LNet: Removed LNI 192.168.202.134@tcp [ 3326.258408] Key type .llcrypt unregistered [ 3326.260339] Key type ._llcrypt unregistered [ 3336.966405] Key type ._llcrypt registered [ 3336.968133] Key type .llcrypt registered [ 3337.018440] Lustre: DEBUG MARKER: oleg234-server.virtnet: executing load_modules_local [ 3337.643315] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 3337.662046] alg: No test for adler32 (adler32-zlib) [ 3338.619323] Lustre: Lustre: Build Version: 2.17.50_65_g69bb343 [ 3338.745747] LNet: Added LNI 192.168.202.134@tcp [8/256/0/180] [ 3340.367132] Key type lgssc registered [ 3340.876872] Lustre: Echo OBD driver; http://www.lustre.org/ [ 3344.469925] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 3344.477562] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 3345.665803] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 3347.352428] Lustre: DEBUG MARKER: oleg234-server.virtnet: executing set_default_debug -1 all [ 3351.144875] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 3351.317720] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 3353.125418] Lustre: DEBUG MARKER: oleg234-server.virtnet: executing set_default_debug -1 all [ 3355.707741] Lustre: DEBUG MARKER: oleg234-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 3357.449911] Lustre: DEBUG MARKER: oleg234-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 3360.557042] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 3360.702911] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 3361.769551] Lustre: lustre-OST0000: new connection from lustre-MDT0001-mdtlov (cleaning up unused objects from 0x280000400:72 to 0x280000400:97) [ 3363.108987] Lustre: DEBUG MARKER: oleg234-server.virtnet: executing set_default_debug -1 all [ 3365.400642] Lustre: DEBUG MARKER: oleg234-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 3370.983816] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:85 to 0x280000401:129) [ 3378.469625] Lustre: 136630:0:(mdt_handler.c:64:mdt_max_mod_rpcs_per_client_set()) max_mod_rpcs_per_client is deprecated, set mdt.*.max_mod_rpcs_in_flight parameter instead [ 3379.229602] Lustre: 136679:0:(mdt_handler.c:64:mdt_max_mod_rpcs_per_client_set()) max_mod_rpcs_per_client is deprecated, set mdt.*.max_mod_rpcs_in_flight parameter instead [ 3379.816308] Lustre: Modifying parameter lustre.mdc.lustre-MDT0000-mdc-*.max_rpcs_in_flight=8 in log params [ 3381.141944] Lustre: 136824:0:(mgs_llog.c:1348:mgs_modify_param()) MGS: modify lustre/mdc.lustre-MDT0000-mdc-*.max_rpcs_in_flight=8 (mode = 0) failed: rc = -17 [ 3381.531268] Lustre: Modifying parameter lustre.mdc.lustre-MDT0000-mdc-*.max_mod_rpcs_in_flight=7 in log params [ 3381.533701] Lustre: Skipped 1 previous similar message [ 3381.937890] Lustre: 136922:0:(mdt_handler.c:64:mdt_max_mod_rpcs_per_client_set()) max_mod_rpcs_per_client is deprecated, set mdt.*.max_mod_rpcs_in_flight parameter instead [ 3385.198119] LustreError: 137020:0:(obd_class.h:479:obd_check_dev()) Device 21 not setup [ 3385.247125] Lustre: server umount lustre-OST0000 complete [ 3387.360142] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 3387.363043] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 3387.368598] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 3391.456382] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 3391.457306] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 3396.576862] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 3396.579253] Lustre: Skipped 2 previous similar messages [ 3396.580573] LustreError: 137228:0:(obd_class.h:479:obd_check_dev()) Device 9 not setup [ 3396.583719] LustreError: 137228:0:(obd_class.h:479:obd_check_dev()) Skipped 3 previous similar messages [ 3396.610370] Lustre: server umount lustre-MDT0000 complete [ 3398.217069] LustreError: 135081:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1770688153 with bad export cookie 15520034048567532782 [ 3398.221150] LustreError: MGC192.168.202.134@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 3398.222303] LustreError: 135081:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 3407.334842] LustreError: 137430:0:(obd_class.h:479:obd_check_dev()) Device 16 not setup [ 3407.339789] LustreError: 137430:0:(obd_class.h:479:obd_check_dev()) Skipped 5 previous similar messages [ 3407.402674] Lustre: server umount lustre-MDT0001 complete [ 3411.732417] Lustre: DEBUG MARKER: oleg234-server.virtnet: executing unload_modules_local [ 3413.089993] Key type lgssc unregistered [ 3413.231691] LNet: 137962:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 3413.236480] LNetError: 137962:0:(acceptor.c:264:lnet_acceptor_remove_socket()) Interface ens2 not found [ 3413.244746] LNet: Removed LNI 192.168.202.134@tcp [ 3413.612137] Key type .llcrypt unregistered [ 3413.614055] Key type ._llcrypt unregistered [ 3425.350153] Key type ._llcrypt registered [ 3425.351654] Key type .llcrypt registered [ 3425.408754] Lustre: DEBUG MARKER: oleg234-server.virtnet: executing load_modules_local [ 3425.900736] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 3425.908851] alg: No test for adler32 (adler32-zlib) [ 3426.816579] Lustre: Lustre: Build Version: 2.17.50_65_g69bb343 [ 3426.956825] LNet: Added LNI 192.168.202.134@tcp [8/256/0/180] [ 3428.567125] Key type lgssc registered [ 3429.077971] Lustre: Echo OBD driver; http://www.lustre.org/ [ 3432.828399] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 3432.835874] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 3434.047844] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 3435.716740] Lustre: DEBUG MARKER: oleg234-server.virtnet: executing set_default_debug -1 all [ 3439.536054] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 3439.725435] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 3441.391610] Lustre: DEBUG MARKER: oleg234-server.virtnet: executing set_default_debug -1 all [ 3443.685197] Lustre: DEBUG MARKER: oleg234-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 3445.247128] Lustre: DEBUG MARKER: oleg234-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 3448.313039] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 3448.504851] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 3449.515937] Lustre: lustre-OST0000: new connection from lustre-MDT0001-mdtlov (cleaning up unused objects from 0x280000400:72 to 0x280000400:129) [ 3451.031784] Lustre: DEBUG MARKER: oleg234-server.virtnet: executing set_default_debug -1 all [ 3453.542325] Lustre: DEBUG MARKER: oleg234-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 3458.728277] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:85 to 0x280000401:161) [ 3462.324716] Lustre: *** cfs_fail_loc=159, val=0*** [ 3477.946119] Lustre: lustre-MDT0000: Client 1e97633d-7660-4851-81cd-a5c24371583f (at 192.168.202.34@tcp) reconnecting [ 3479.521172] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 3479.521716] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 3479.522578] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 3479.526196] Lustre: Skipped 1 previous similar message [ 3484.640975] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 3484.644192] Lustre: Skipped 2 previous similar messages [ 3485.614109] LustreError: 141708:0:(obd_class.h:479:obd_check_dev()) Device 21 not setup [ 3485.668926] Lustre: server umount lustre-OST0000 complete [ 3489.760744] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 3489.764986] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 3489.766163] Lustre: Skipped 1 previous similar message [ 3493.380893] LustreError: 141909:0:(obd_class.h:479:obd_check_dev()) Device 9 not setup [ 3493.383221] LustreError: 141909:0:(obd_class.h:479:obd_check_dev()) Skipped 3 previous similar messages [ 3493.423046] Lustre: server umount lustre-MDT0000 complete [ 3494.879980] LustreError: 139334:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 3494.887049] LustreError: 139334:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 3 previous similar messages [ 3494.899433] LustreError: 139316:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1770688250 with bad export cookie 9924528516418887175 [ 3494.900864] LustreError: MGC192.168.202.134@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 3494.904231] LustreError: 139316:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 3494.980673] LustreError: 142111:0:(obd_class.h:479:obd_check_dev()) Device 16 not setup [ 3494.982715] LustreError: 142111:0:(obd_class.h:479:obd_check_dev()) Skipped 5 previous similar messages [ 3495.040882] Lustre: server umount lustre-MDT0001 complete [ 3499.209457] Lustre: DEBUG MARKER: oleg234-server.virtnet: executing unload_modules_local [ 3500.385917] Key type lgssc unregistered [ 3500.543598] LNet: 142642:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 3500.546963] LNetError: 142642:0:(acceptor.c:264:lnet_acceptor_remove_socket()) Interface ens2 not found [ 3500.559429] LNet: Removed LNI 192.168.202.134@tcp [ 3500.912100] Key type .llcrypt unregistered [ 3500.913757] Key type ._llcrypt unregistered [ 3512.330802] Key type ._llcrypt registered [ 3512.332348] Key type .llcrypt registered [ 3512.375243] Lustre: DEBUG MARKER: oleg234-server.virtnet: executing load_modules_local [ 3512.746069] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 3512.769668] alg: No test for adler32 (adler32-zlib) [ 3513.655883] Lustre: Lustre: Build Version: 2.17.50_65_g69bb343 [ 3513.751772] LNet: Added LNI 192.168.202.134@tcp [8/256/0/180] [ 3515.343213] Key type lgssc registered [ 3515.834958] Lustre: Echo OBD driver; http://www.lustre.org/ [ 3519.177900] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 3519.190971] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 3520.416812] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 3522.092881] Lustre: DEBUG MARKER: oleg234-server.virtnet: executing set_default_debug -1 all [ 3526.617711] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 3526.812226] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 3528.420722] Lustre: DEBUG MARKER: oleg234-server.virtnet: executing set_default_debug -1 all [ 3530.740400] Lustre: DEBUG MARKER: oleg234-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 3532.142391] Lustre: DEBUG MARKER: oleg234-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 3535.180978] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 3535.357820] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 3536.363332] Lustre: lustre-OST0000: new connection from lustre-MDT0001-mdtlov (cleaning up unused objects from 0x280000400:72 to 0x280000400:161) [ 3537.842522] Lustre: DEBUG MARKER: oleg234-server.virtnet: executing set_default_debug -1 all [ 3540.218817] Lustre: DEBUG MARKER: oleg234-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 3545.580734] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:170 to 0x280000401:193) [ 3564.350551] Lustre: lustre-OST0000: evicting 9df0cd2e-ca51-49d6-a528-a7482a2bafc3 (at 192.168.202.34@tcp) by administrative request [ 3564.355073] Lustre: Skipped 1 previous similar message [ 3591.650300] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 3591.658494] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 3591.658532] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 3591.668812] Lustre: Skipped 1 previous similar message [ 3594.799733] LustreError: 146812:0:(obd_class.h:479:obd_check_dev()) Device 21 not setup [ 3594.848533] Lustre: server umount lustre-OST0000 complete [ 3596.768745] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 3596.772701] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 3596.775939] Lustre: Skipped 2 previous similar messages [ 3601.888417] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 3601.892487] Lustre: Skipped 2 previous similar messages [ 3602.502413] LustreError: 147013:0:(obd_class.h:479:obd_check_dev()) Device 9 not setup [ 3602.504871] LustreError: 147013:0:(obd_class.h:479:obd_check_dev()) Skipped 3 previous similar messages [ 3602.540701] Lustre: server umount lustre-MDT0000 complete [ 3604.053902] LustreError: 143995:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1770688359 with bad export cookie 6993266817798453885 [ 3604.057345] LustreError: MGC192.168.202.134@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 3604.058434] LustreError: 143995:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 3604.144062] LustreError: 147215:0:(obd_class.h:479:obd_check_dev()) Device 16 not setup [ 3604.146363] LustreError: 147215:0:(obd_class.h:479:obd_check_dev()) Skipped 5 previous similar messages [ 3604.206962] Lustre: server umount lustre-MDT0001 complete [ 3608.306503] Lustre: DEBUG MARKER: oleg234-server.virtnet: executing unload_modules_local [ 3609.439046] Key type lgssc unregistered [ 3609.585693] LNet: 147746:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 3609.589113] LNetError: 147746:0:(acceptor.c:264:lnet_acceptor_remove_socket()) Interface ens2 not found [ 3609.601167] LNet: Removed LNI 192.168.202.134@tcp [ 3609.960809] Key type .llcrypt unregistered [ 3609.962417] Key type ._llcrypt unregistered [ 3639.510049] Key type ._llcrypt registered [ 3639.511349] Key type .llcrypt registered [ 3639.553189] Lustre: DEBUG MARKER: oleg234-server.virtnet: executing load_modules_local [ 3639.925348] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 3639.982262] alg: No test for adler32 (adler32-zlib) [ 3640.878770] Lustre: Lustre: Build Version: 2.17.50_65_g69bb343 [ 3640.975162] LNet: Added LNI 192.168.202.134@tcp [8/256/0/180] [ 3642.583578] Key type lgssc registered [ 3643.038402] Lustre: Echo OBD driver; http://www.lustre.org/ [ 3646.364458] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 3646.378386] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 3647.595716] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 3649.217921] Lustre: DEBUG MARKER: oleg234-server.virtnet: executing set_default_debug -1 all [ 3653.000231] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 3653.249334] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 3654.987270] Lustre: DEBUG MARKER: oleg234-server.virtnet: executing set_default_debug -1 all [ 3657.202783] Lustre: DEBUG MARKER: oleg234-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 3658.729314] Lustre: DEBUG MARKER: oleg234-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 3661.779694] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 3661.978580] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 3663.017044] Lustre: lustre-OST0000: new connection from lustre-MDT0001-mdtlov (cleaning up unused objects from 0x280000400:72 to 0x280000400:193) [ 3664.257355] Lustre: DEBUG MARKER: oleg234-server.virtnet: executing set_default_debug -1 all [ 3666.441134] Lustre: DEBUG MARKER: oleg234-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 3673.255378] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:195 to 0x280000401:225) [ 3683.808569] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 3683.815628] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 3683.820983] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 3683.824136] Lustre: Skipped 1 previous similar message [ 3686.129096] LustreError: 152458:0:(obd_class.h:479:obd_check_dev()) Device 21 not setup [ 3686.172648] Lustre: server umount lustre-OST0000 complete [ 3688.928095] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 3688.930203] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 3688.930800] LustreError: Skipped 1 previous similar message [ 3688.935499] Lustre: Skipped 1 previous similar message [ 3688.936317] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 3688.941715] Lustre: Skipped 1 previous similar message [ 3693.763132] LustreError: 152660:0:(obd_class.h:479:obd_check_dev()) Device 9 not setup [ 3693.766060] LustreError: 152660:0:(obd_class.h:479:obd_check_dev()) Skipped 3 previous similar messages [ 3693.802650] Lustre: server umount lustre-MDT0000 complete [ 3694.049261] LustreError: 150188:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 3694.055917] LustreError: 150188:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 3 previous similar messages [ 3695.378272] LustreError: 152358:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1770688451 with bad export cookie 5638992419486637308 [ 3695.381109] LustreError: MGC192.168.202.134@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 3695.382892] LustreError: 152358:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 3695.458417] LustreError: 152861:0:(obd_class.h:479:obd_check_dev()) Device 16 not setup [ 3695.461088] LustreError: 152861:0:(obd_class.h:479:obd_check_dev()) Skipped 5 previous similar messages [ 3695.516709] Lustre: server umount lustre-MDT0001 complete [ 3699.575524] Lustre: DEBUG MARKER: oleg234-server.virtnet: executing unload_modules_local [ 3700.700610] Key type lgssc unregistered [ 3700.829551] LNet: 153392:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 3700.832569] LNetError: 153392:0:(acceptor.c:264:lnet_acceptor_remove_socket()) Interface ens2 not found [ 3700.844363] LNet: Removed LNI 192.168.202.134@tcp [ 3701.175329] Key type .llcrypt unregistered [ 3701.176647] Key type ._llcrypt unregistered [ 3710.883259] Key type ._llcrypt registered [ 3710.884662] Key type .llcrypt registered [ 3710.923878] Lustre: DEBUG MARKER: oleg234-server.virtnet: executing load_modules_local [ 3711.340345] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 3711.352441] alg: No test for adler32 (adler32-zlib) [ 3712.227492] Lustre: Lustre: Build Version: 2.17.50_65_g69bb343 [ 3712.324916] LNet: Added LNI 192.168.202.134@tcp [8/256/0/180] [ 3713.911147] Key type lgssc registered [ 3714.331527] Lustre: Echo OBD driver; http://www.lustre.org/ [ 3719.834542] Lustre: DEBUG MARKER: == conf-sanity test 100: check lshowmount lists MGS, MDT, OST and 0@lo ========================================================== 20:54:34 (1770688474) [ 3725.790085] Lustre: DEBUG MARKER: oleg234-server.virtnet: executing set_hostid [ 3729.822474] Lustre: DEBUG MARKER: oleg234-server.virtnet: executing load_modules_local [ 3745.397331] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 3745.407499] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 3746.543656] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 3746.559797] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 3746.605476] Lustre: lustre-MDT0000: new disk, initializing [ 3746.644557] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 3746.653213] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 3748.452246] Lustre: DEBUG MARKER: oleg234-server.virtnet: executing set_default_debug -1 all [ 3753.436078] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 3753.484524] Lustre: 157577:0:(mgs_llog.c:1348:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 3753.511190] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [ 3753.513366] Lustre: Skipped 1 previous similar message [ 3753.557820] Lustre: lustre-MDT0001: new disk, initializing [ 3753.599518] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 3753.617522] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [ 3753.621695] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 3755.285499] Lustre: DEBUG MARKER: oleg234-server.virtnet: executing set_default_debug -1 all [ 3758.982468] Lustre: DEBUG MARKER: oleg234-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 3760.422289] Lustre: DEBUG MARKER: oleg234-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 3763.327347] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 3763.443942] Lustre: lustre-OST0000: new disk, initializing [ 3763.446765] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 3763.450878] Lustre: 159316:0:(osd_compat.c:1354:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 3763.486362] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 3764.552666] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [ 3764.557373] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 3764.606711] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 3765.845661] Lustre: DEBUG MARKER: oleg234-server.virtnet: executing set_default_debug -1 all [ 3769.610955] Lustre: DEBUG MARKER: oleg234-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 3774.945699] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 3774.946492] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 3774.950496] LustreError: Skipped 1 previous similar message [ 3774.951884] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 3774.962380] Lustre: Skipped 1 previous similar message [ 3780.069244] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 3780.072750] Lustre: Skipped 2 previous similar messages [ 3780.654137] LustreError: 160155:0:(obd_class.h:479:obd_check_dev()) Device 19 not setup [ 3780.706788] Lustre: server umount lustre-OST0000 complete [ 3784.671852] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 3784.674994] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 3784.680206] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 3788.427600] LustreError: 160356:0:(obd_class.h:479:obd_check_dev()) Device 22 not setup [ 3788.430885] LustreError: 160356:0:(obd_class.h:479:obd_check_dev()) Skipped 3 previous similar messages [ 3788.469878] Lustre: server umount lustre-MDT0000 complete [ 3789.955511] LustreError: 157569:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1770688545 with bad export cookie 3569139359129095356 [ 3789.957914] LustreError: MGC192.168.202.134@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 3789.960524] LustreError: 157569:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) Skipped 2 previous similar messages [ 3790.033329] LustreError: 160558:0:(obd_class.h:479:obd_check_dev()) Device 21 not setup [ 3790.035856] LustreError: 160558:0:(obd_class.h:479:obd_check_dev()) Skipped 5 previous similar messages [ 3790.107431] Lustre: server umount lustre-MDT0001 complete [ 3794.423413] Lustre: DEBUG MARKER: oleg234-server.virtnet: executing unload_modules_local [ 3795.594637] Key type lgssc unregistered [ 3795.740352] LNet: 161089:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 3795.743491] LNetError: 161089:0:(acceptor.c:264:lnet_acceptor_remove_socket()) Interface ens2 not found [ 3795.750302] LNet: Removed LNI 192.168.202.134@tcp [ 3796.065716] Key type .llcrypt unregistered [ 3796.067229] Key type ._llcrypt unregistered [ 3805.734787] Key type ._llcrypt registered [ 3805.736296] Key type .llcrypt registered [ 3805.775387] Lustre: DEBUG MARKER: oleg234-server.virtnet: executing load_modules_local [ 3806.233949] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 3806.263916] alg: No test for adler32 (adler32-zlib) [ 3807.134639] Lustre: Lustre: Build Version: 2.17.50_65_g69bb343 [ 3807.220881] LNet: Added LNI 192.168.202.134@tcp [8/256/0/180] [ 3808.807202] Key type lgssc registered [ 3809.216958] Lustre: Echo OBD driver; http://www.lustre.org/ [ 3812.158977] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 3812.168316] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 3813.359260] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 3814.887327] Lustre: DEBUG MARKER: oleg234-server.virtnet: executing set_default_debug -1 all [ 3818.482076] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 3818.635470] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 3820.219284] Lustre: DEBUG MARKER: oleg234-server.virtnet: executing set_default_debug -1 all [ 3822.359035] Lustre: DEBUG MARKER: oleg234-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 3823.721346] Lustre: DEBUG MARKER: oleg234-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 3826.266731] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 3826.384000] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 3828.376642] Lustre: DEBUG MARKER: oleg234-server.virtnet: executing set_default_debug -1 all [ 3830.389380] Lustre: DEBUG MARKER: oleg234-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 3838.694662] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:3 to 0x280000401:33) [ 3844.495860] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 3844.506249] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 3844.511664] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 3844.515194] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3844.519719] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 3844.861608] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3845.197932] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 3845.200547] Lustre: Skipped 1 previous similar message [ 3845.208037] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 3845.212536] Lustre: Skipped 1 previous similar message [ 3845.214595] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 3845.217729] Lustre: Skipped 1 previous similar message [ 3845.220946] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3845.225447] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 3845.229257] Lustre: Skipped 1 previous similar message [ 3845.545165] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3845.867781] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3846.192656] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3846.551612] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 3846.554141] Lustre: Skipped 3 previous similar messages [ 3846.560632] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 3846.565901] Lustre: Skipped 3 previous similar messages [ 3846.567851] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 3846.571390] Lustre: Skipped 3 previous similar messages [ 3846.573157] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3846.577761] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 3846.580960] Lustre: Skipped 3 previous similar messages [ 3846.906313] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3847.247863] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3847.564972] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3847.955110] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3848.301543] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3848.627039] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 3848.629211] Lustre: Skipped 5 previous similar messages [ 3848.635215] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 3848.639208] Lustre: Skipped 5 previous similar messages [ 3848.640940] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 3848.642822] Lustre: Skipped 5 previous similar messages [ 3848.644469] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3848.648085] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 3848.650550] Lustre: Skipped 5 previous similar messages [ 3848.986501] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3849.341627] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3849.693145] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3850.055471] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3850.436297] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3850.808274] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3851.172464] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3851.576508] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3851.981332] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3852.412651] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3852.806333] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 3852.809554] Lustre: Skipped 10 previous similar messages [ 3852.816654] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 3852.824118] Lustre: Skipped 10 previous similar messages [ 3852.826655] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 3852.830508] Lustre: Skipped 10 previous similar messages [ 3852.832899] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3852.840756] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 3852.843679] Lustre: Skipped 10 previous similar messages [ 3853.265644] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3853.621089] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3853.989086] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3854.385517] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3854.793749] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3855.219094] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3855.615032] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3855.999686] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3856.393307] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3856.823659] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3857.197583] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3857.641432] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3858.105172] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3858.512403] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3858.932448] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3859.315942] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3859.715818] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3860.121287] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3860.498111] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3860.880201] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 3860.882895] Lustre: Skipped 19 previous similar messages [ 3860.888794] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 3860.893391] Lustre: Skipped 19 previous similar messages [ 3860.895463] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 3860.898674] Lustre: Skipped 19 previous similar messages [ 3860.900619] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3860.904843] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 3860.907392] Lustre: Skipped 19 previous similar messages [ 3861.289847] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3861.634740] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3862.031662] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3862.414292] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3862.824073] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3863.211417] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3863.647227] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3864.033067] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3864.397239] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3864.784038] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3865.163346] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3865.559405] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3865.957630] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3866.343090] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3866.723990] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3867.082058] LustreError: 162479:0:(osp_precreate.c:651:osp_precreate_send()) lustre-OST0000-osc-MDT0000: can't precreate: rc = -5 [ 3867.085211] LustreError: 162479:0:(osp_precreate.c:1355:osp_precreate_thread()) lustre-OST0000-osc-MDT0000: cannot precreate objects: rc = -5 [ 3867.088112] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3867.462367] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3867.826885] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3868.227588] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3868.613795] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3869.006883] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3869.405913] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3869.789986] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3870.176183] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3870.568956] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3870.975375] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3871.381694] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3871.835635] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3872.210525] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3872.595418] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3872.994034] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3873.362925] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3873.751659] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3874.108712] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3874.474772] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3874.815880] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3875.185545] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3875.577567] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3875.960812] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3876.346959] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3876.730465] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3877.138198] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 3877.141690] Lustre: Skipped 41 previous similar messages [ 3877.148199] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 3877.152642] Lustre: Skipped 41 previous similar messages [ 3877.155302] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 3877.158149] Lustre: Skipped 41 previous similar messages [ 3877.160231] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3877.164428] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 3877.166995] Lustre: Skipped 41 previous similar messages [ 3877.548817] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3877.888500] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3878.223517] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3878.552510] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3878.988098] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3879.336912] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3879.723867] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3880.132587] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3880.478927] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3880.853227] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3881.181675] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3881.555457] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3881.921358] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3882.287683] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3882.670696] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3883.057627] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3883.431486] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3883.810024] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3884.179055] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3884.538329] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3884.907381] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3885.260144] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3885.605913] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3885.951979] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3886.322097] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3886.671653] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3887.066614] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3887.419955] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3887.828931] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3888.254256] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3888.590565] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3888.963754] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3889.338242] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3889.685386] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3890.043623] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3890.399775] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3890.752485] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3891.092344] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3891.431165] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3891.767981] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3892.146524] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3892.517861] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3892.882502] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3893.263990] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3893.681820] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3894.036986] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3894.403373] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3894.784183] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3895.146771] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3895.518063] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3895.883973] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3896.289419] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3896.658240] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3897.039490] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3897.442514] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3897.831493] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3898.193602] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3898.575429] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3898.961329] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3899.332156] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3899.675110] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3900.016640] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3900.381388] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3900.749091] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3901.101511] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3901.461231] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3901.844857] LustreError: 162479:0:(osp_precreate.c:651:osp_precreate_send()) lustre-OST0000-osc-MDT0000: can't precreate: rc = -5 [ 3901.848800] LustreError: 162479:0:(osp_precreate.c:1355:osp_precreate_thread()) lustre-OST0000-osc-MDT0000: cannot precreate objects: rc = -5 [ 3901.854488] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3902.229268] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3902.631404] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3903.027426] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3903.411400] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3903.819520] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3904.178432] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3904.588643] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3904.979435] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3905.405174] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3905.813412] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3906.186538] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3906.572829] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3906.925104] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3907.306861] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3907.676186] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3908.060078] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3908.409301] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3908.807711] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3909.170706] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 3909.173670] Lustre: Skipped 85 previous similar messages [ 3909.181229] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 3909.186647] Lustre: Skipped 85 previous similar messages [ 3909.188986] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 3909.191652] Lustre: Skipped 85 previous similar messages [ 3909.193820] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3909.201631] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 3909.203611] Lustre: Skipped 85 previous similar messages [ 3909.554786] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3909.891269] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3910.211877] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3910.531650] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3910.877161] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3911.237086] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3911.610972] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3911.958236] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3912.316831] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3912.720334] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3913.115040] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3913.479303] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3913.873015] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3914.225235] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3914.557979] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3914.876083] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3915.189170] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3915.512364] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3915.893321] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3916.237346] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3916.596943] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3916.930592] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3917.272091] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3917.601968] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3917.944684] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3918.343974] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3918.725091] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3919.089680] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3919.465786] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3919.846975] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3920.181282] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3920.508257] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3920.876466] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3921.224222] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3921.572589] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3921.928546] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3922.296781] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3922.655398] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3923.051015] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3923.471936] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3923.852218] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3924.230092] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3924.602489] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3924.989370] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3925.319205] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3925.645709] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3925.971849] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3926.305682] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3926.659914] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3927.020585] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3927.361892] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3927.701781] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3928.096594] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3928.424091] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3928.767827] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3929.176468] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3929.508475] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3929.868874] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3930.248671] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3930.599651] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3930.916715] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3931.242942] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3931.582704] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3931.919356] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3932.266559] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3932.607239] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3932.963018] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3933.358445] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3933.758385] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3934.140060] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3934.493916] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3934.844057] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3935.192733] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3935.593830] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3935.925636] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3936.267761] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3936.613830] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3936.971527] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3937.306400] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3937.626228] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3937.951514] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3938.316913] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3938.681721] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3939.023567] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3939.374248] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3939.716276] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3940.069476] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3940.421917] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3940.777259] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3941.118431] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3941.489495] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3941.847621] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3942.220968] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3942.567920] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3942.913425] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3943.289773] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3943.663808] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3944.026866] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3944.401544] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3944.781488] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3945.120291] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3945.495202] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3945.874745] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3946.222135] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3946.578331] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3946.923123] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3947.300866] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3947.634550] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3948.006789] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3948.354653] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3948.716571] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3949.076066] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3949.403785] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3949.768125] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3950.116903] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3950.453564] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3950.793672] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3951.120263] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3951.498514] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3951.861731] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3952.227472] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3952.595089] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3952.941733] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3953.308672] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3953.665719] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3954.020078] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3954.383868] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3954.763182] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3955.114327] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3955.475315] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3955.850258] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3956.191305] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3956.535765] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3956.921638] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3957.274456] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3957.626173] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3957.982379] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3958.332564] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3958.699431] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3959.064777] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3959.424306] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3959.775426] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3960.119952] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3960.507543] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3960.870600] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3961.224444] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3961.582958] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3961.927033] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3962.303952] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3962.636204] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3962.964113] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3963.302689] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3963.620388] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3963.956160] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3964.300301] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3964.676079] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3965.044101] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3965.383623] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3965.770235] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3966.135536] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3966.498283] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3966.825183] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3967.139556] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3967.515324] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3967.925789] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3968.297920] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3968.700469] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3969.062098] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3969.437753] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3969.791059] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3970.182127] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3970.571884] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3970.971353] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3971.423284] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3971.791826] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3972.219619] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3972.598813] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3972.987413] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3973.358383] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 3973.361408] Lustre: Skipped 178 previous similar messages [ 3973.367598] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 3973.372772] Lustre: Skipped 178 previous similar messages [ 3973.374601] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 3973.377220] Lustre: Skipped 178 previous similar messages [ 3973.378992] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3973.383655] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 3973.386091] Lustre: Skipped 178 previous similar messages [ 3973.734325] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3974.137356] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3974.496733] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3974.911933] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3975.347620] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3975.725707] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3976.131136] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3976.525488] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3976.931841] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3977.349339] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3977.731836] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3978.158670] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3978.640665] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3979.069640] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3979.524692] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3979.936533] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3980.337503] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3980.777193] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3981.208856] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3981.659557] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3982.048774] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3982.484732] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3982.890881] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3983.322349] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3983.767619] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3984.125863] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3984.497926] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3984.926354] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3985.350849] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3985.772512] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3986.205110] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3986.627974] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3987.028976] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3987.412705] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3987.838723] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3988.197081] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3988.539652] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3988.891386] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3989.290302] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3989.676550] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3990.120536] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3990.489848] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3990.907038] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3991.296472] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3991.674858] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3992.084668] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3992.437088] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3992.853776] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3993.249337] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3993.679237] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3994.094051] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3994.494040] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3994.916056] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3995.284914] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3995.668619] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3996.049584] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3996.564168] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3997.004572] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3997.432939] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3997.905721] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3998.321360] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3998.736741] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3999.144405] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3999.545241] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4000.030144] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4000.448520] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4000.884989] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4001.329657] LustreError: 162479:0:(osp_precreate.c:651:osp_precreate_send()) lustre-OST0000-osc-MDT0000: can't precreate: rc = -5 [ 4001.335724] LustreError: 162479:0:(osp_precreate.c:1355:osp_precreate_thread()) lustre-OST0000-osc-MDT0000: cannot precreate objects: rc = -5 [ 4001.337236] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4001.762885] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4002.206962] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4002.622570] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4003.064847] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4003.512785] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4003.896670] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4004.317952] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4004.784124] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4005.208221] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4005.683965] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4006.123396] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4006.568304] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4007.075184] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4007.528505] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4007.958892] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4008.407596] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4008.892100] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4009.375736] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4009.838794] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4010.299277] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4010.802780] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4011.281984] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4011.743866] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4012.242217] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4012.729263] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4013.208717] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4013.683393] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4014.108423] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4014.563475] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4015.027397] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4015.474654] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4015.931252] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4016.409370] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4016.877132] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4017.369375] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4017.859726] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4018.349136] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4018.882150] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4019.413703] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4019.858450] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4020.315563] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4020.764489] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4021.229113] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4021.721078] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4022.205775] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4022.647518] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4023.069744] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4023.508609] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4023.940723] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4024.373677] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4024.745856] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4025.232683] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4025.629773] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4026.032607] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4026.444217] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4026.870895] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4027.301125] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4027.729290] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4028.105372] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4028.499312] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4028.918450] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4029.316860] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4029.740157] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4030.128748] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4030.494541] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4030.885727] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4031.329720] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4031.718089] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4032.111639] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4032.590883] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4032.941668] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4033.286579] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4033.633321] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4034.019063] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4034.369492] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4034.720217] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4035.035301] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4035.405342] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4035.776761] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4036.107249] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4036.451171] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4036.792996] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4037.166708] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4037.569799] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4037.978661] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4038.357211] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4038.737744] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4039.106021] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4039.513889] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4039.884173] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4040.286602] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4040.703384] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4041.066456] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4041.429906] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4041.827620] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4042.194780] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4042.584923] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4042.992039] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4043.391527] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4043.779506] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4044.158315] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4044.622188] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4045.050071] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4045.464149] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4045.921796] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4046.357518] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4046.765196] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4047.160038] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4047.562231] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4047.998166] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4048.370802] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4048.762195] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4049.208536] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4049.617224] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4050.009377] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4050.417068] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4050.785119] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4051.133831] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4051.541726] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4051.944670] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4052.323345] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4052.723823] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4053.159077] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4053.522264] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4053.899262] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4054.291089] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4054.642421] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4055.039987] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4055.416078] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4055.766989] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4056.142280] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4056.492061] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4056.858493] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4057.199076] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4057.550796] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4057.915902] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4058.272443] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4058.631971] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4058.980831] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4059.323082] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4059.677358] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4060.122818] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4060.508638] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4060.898744] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4061.298481] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4061.684596] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4062.095809] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4062.538108] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4062.898301] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4063.309356] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4063.667804] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4064.023564] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4064.383823] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4064.714274] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4065.108344] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4065.432472] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4065.764720] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4066.111395] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4066.460600] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4066.815230] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4067.169614] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4067.536971] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4067.884964] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4068.252352] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4068.628644] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4069.009065] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4069.365022] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4069.720704] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4070.065436] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4070.412511] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4070.761217] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4071.138825] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4071.483841] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4071.899167] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4072.317481] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4072.708820] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4073.081646] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4073.437703] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4073.799947] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4074.128235] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4074.492605] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4074.885409] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4075.260185] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4075.598756] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4075.973169] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4076.328749] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4076.714846] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4077.097718] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4077.468820] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4077.848930] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4078.176876] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4078.625387] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4078.952684] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4079.303910] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4079.679294] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4080.051716] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4080.420577] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4080.764918] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4081.111751] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4081.500697] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4081.864434] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4082.281941] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4082.720648] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4083.078623] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4083.437981] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4083.809723] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4084.173248] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4084.521756] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4084.890753] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4085.242658] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4085.598620] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4085.928711] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4086.273081] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4086.608595] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4086.960874] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4087.315209] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4087.712720] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4088.072106] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4088.433928] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4088.790033] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4089.167981] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4089.527038] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4089.884217] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4090.217464] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4090.562861] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4090.910526] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4091.253209] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4091.598464] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4092.009493] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4092.371119] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4092.731963] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4093.156721] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4093.507586] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4093.869930] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4094.226451] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4094.589804] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4094.946673] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4095.293594] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4095.621058] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4095.978625] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4096.300643] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4096.642663] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4097.043237] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4097.433028] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4097.840769] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4098.210905] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4098.619461] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4098.997452] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4099.345329] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4099.718516] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4100.061328] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4100.419717] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4100.745604] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4101.085425] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4101.462905] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 4101.465679] Lustre: Skipped 322 previous similar messages [ 4101.471797] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4101.476663] Lustre: Skipped 322 previous similar messages [ 4101.478514] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 4101.481478] Lustre: Skipped 322 previous similar messages [ 4101.483657] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4101.487748] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 4101.490641] Lustre: Skipped 322 previous similar messages [ 4101.812499] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4102.185412] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4102.531450] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4102.869799] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4103.250151] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4103.587147] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4103.919455] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4104.298729] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4104.643244] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4104.984115] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4105.385241] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4105.743568] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4106.135445] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4106.580441] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4107.010663] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4107.430372] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4107.837467] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4108.224658] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4108.600386] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4109.007545] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4109.437427] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4109.814424] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4110.202910] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4110.619272] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4111.007863] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4111.482877] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4111.931171] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4112.379744] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4112.927561] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4113.396753] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4113.819844] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4114.232642] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4114.673482] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4115.101620] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4115.505666] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4115.967027] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4116.359419] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4116.742355] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4117.112095] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4117.561614] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4117.988428] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4118.422819] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4118.845240] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4119.255169] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4119.650930] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4120.074383] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4120.454564] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4120.878397] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4121.242662] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4121.686311] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4122.098142] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4122.464417] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4122.911549] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4123.372522] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4123.791269] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4124.154110] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4124.529982] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4124.887116] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4125.236254] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4125.574373] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4125.948109] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4126.328871] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4126.690873] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4127.074227] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4127.439991] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4127.848306] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4128.250592] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4128.707483] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4129.107959] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4129.491377] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4129.928756] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4130.352519] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4130.724293] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4131.129475] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4131.487252] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4131.891413] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4132.251689] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4132.658570] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4133.110198] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4133.505147] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4133.907930] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4134.277454] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4134.631378] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4134.993711] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4135.394466] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4135.723076] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4136.081805] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4136.403160] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4136.730440] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4137.088488] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4137.428397] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4137.777943] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4138.117835] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4138.448277] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4138.798683] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4139.124850] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4139.448218] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4139.805498] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4140.139152] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4140.468074] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4140.797046] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4141.126169] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4141.455915] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4141.801871] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4142.121077] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4142.436361] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4142.786372] LustreError: 162479:0:(osp_precreate.c:651:osp_precreate_send()) lustre-OST0000-osc-MDT0000: can't precreate: rc = -5 [ 4142.790147] LustreError: 162479:0:(osp_precreate.c:1355:osp_precreate_thread()) lustre-OST0000-osc-MDT0000: cannot precreate objects: rc = -5 [ 4142.791163] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4143.111853] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4143.467796] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4143.812522] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4144.170773] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4144.539651] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4144.902320] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4145.317035] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4145.635459] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4145.956976] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4146.342596] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4146.674699] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4146.994859] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4147.334834] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4147.666675] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4147.980832] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4148.307390] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4148.637430] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4148.969104] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4149.299374] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4149.667648] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4150.064833] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4150.442531] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4150.829439] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4151.173556] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4151.509642] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4151.832797] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4152.165036] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4152.535131] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4152.930060] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4153.349897] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4153.762877] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4154.089017] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4154.443047] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4154.791344] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4155.169087] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4155.563330] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4155.933796] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4156.296098] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4156.657974] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4157.036474] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4157.378520] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4157.742781] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4158.083937] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4158.418102] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4158.771125] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4159.103708] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4159.471654] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4159.825676] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4160.144878] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4160.472785] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4160.821468] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4161.172963] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4161.555573] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4161.897061] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4162.251155] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4162.595797] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4162.979127] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4163.318300] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4163.671578] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4164.043448] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4164.393963] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4164.728203] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4165.073169] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4165.471305] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4165.840892] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4166.176769] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4166.556569] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4166.925848] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4167.267892] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4167.664408] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4168.060157] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4168.421103] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4168.768279] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4169.141036] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4169.493240] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4169.845034] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4170.190954] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4170.540627] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4170.889720] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4171.233469] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4171.586339] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4171.933454] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4172.296572] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4172.681983] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4173.048317] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4173.399742] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4173.766155] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4174.115241] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4174.470591] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4174.848406] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4175.187215] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4175.514223] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4175.869577] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4176.204131] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4176.539828] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4176.885241] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4177.241317] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4177.598626] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4177.983274] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4178.373831] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4178.753753] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4179.162094] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4179.549968] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4179.967049] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4180.329181] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4180.691283] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4181.049740] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4181.394828] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4181.726095] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4182.079501] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4182.415449] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4182.745593] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4183.115507] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4183.465272] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4183.810298] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4184.142653] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4184.494426] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4184.852132] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4185.213743] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4185.582395] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4185.971224] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4186.309507] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4186.658460] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4187.016804] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4187.372987] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4187.744024] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4188.195413] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4188.588717] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4188.964933] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4189.342669] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4189.687746] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4190.043655] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4190.389401] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4190.755768] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4191.129114] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4191.507871] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4191.862133] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4192.220084] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4192.597219] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4193.005212] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4193.407167] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4193.788392] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4194.203045] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4194.576677] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4195.038382] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4195.437509] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4195.802774] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4196.224285] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4196.554925] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4196.949292] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4197.310730] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4197.646075] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4198.009363] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4198.370181] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4198.708571] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4199.076761] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4199.454276] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4199.832396] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4200.234498] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4200.589364] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4200.937085] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4201.314119] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4201.712676] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4202.086340] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4202.436943] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4202.832988] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4203.199408] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4203.536268] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4203.889974] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4204.247766] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4204.588657] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4204.956758] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4205.319285] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4205.700497] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4206.080961] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4206.462686] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4206.840828] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4207.219569] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4207.583775] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4208.003676] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4208.401979] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4208.777885] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4209.130032] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4209.483533] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4209.883345] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4210.285833] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4210.645438] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4210.996069] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4211.344269] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4211.721496] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4212.097329] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4212.530745] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4213.009518] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4213.406656] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4213.811748] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4214.218747] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4214.626405] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4215.059728] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4215.462620] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4215.850278] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4216.224661] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4216.668127] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4217.083299] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4217.479912] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4217.930746] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4218.363342] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4218.782841] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4219.219535] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4219.613782] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4220.068261] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4220.489281] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4220.898666] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4221.316182] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4221.761355] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4222.168146] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4222.602716] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4223.102794] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4223.543229] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4223.989562] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4376.015618] LustreError: 164050:0:(ofd_dev.c:1780:ofd_create_hdl()) lustre-OST0000: unable to precreate: rc = -28 [ 4376.021532] LustreError: 162479:0:(osp_precreate.c:651:osp_precreate_send()) lustre-OST0000-osc-MDT0000: can't precreate: rc = -28 [ 4458.463929] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 4458.467326] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4458.472430] Lustre: Skipped 326 previous similar messages [ 4458.475087] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 4461.032100] LustreError: lustre-OST0000-osc-MDT0000: operation ost_destroy to node 0@lo failed: rc = -107 [ 4461.033697] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 4461.034979] LustreError: Skipped 4 previous similar messages [ 4462.561641] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 4462.566359] Lustre: Skipped 1 previous similar message [ 4463.609199] LustreError: 216236:0:(obd_class.h:479:obd_check_dev()) Device 21 not setup [ 4463.693194] Lustre: server umount lustre-OST0000 complete [ 4467.681799] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 4467.683801] Lustre: Skipped 1 previous similar message [ 4471.169988] LustreError: 216438:0:(obd_class.h:479:obd_check_dev()) Device 9 not setup [ 4471.172229] LustreError: 216438:0:(obd_class.h:479:obd_check_dev()) Skipped 3 previous similar messages [ 4471.264239] Lustre: server umount lustre-MDT0000 complete [ 4472.670480] LustreError: 162439:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1770689228 with bad export cookie 3562359374434494744 [ 4472.674152] LustreError: MGC192.168.202.134@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 4472.674566] LustreError: 162439:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 4472.753995] LustreError: 216639:0:(obd_class.h:479:obd_check_dev()) Device 16 not setup [ 4472.756886] LustreError: 216639:0:(obd_class.h:479:obd_check_dev()) Skipped 5 previous similar messages [ 4472.820490] Lustre: server umount lustre-MDT0001 complete [ 4476.880518] Lustre: DEBUG MARKER: oleg234-server.virtnet: executing unload_modules_local [ 4478.112264] Key type lgssc unregistered [ 4478.271614] LNet: 217170:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 4478.274156] LNetError: 217170:0:(acceptor.c:264:lnet_acceptor_remove_socket()) Interface ens2 not found [ 4478.283473] LNet: Removed LNI 192.168.202.134@tcp [ 4478.585129] Key type .llcrypt unregistered [ 4478.587588] Key type ._llcrypt unregistered [ 4488.652585] Key type ._llcrypt registered [ 4488.653951] Key type .llcrypt registered [ 4488.696592] Lustre: DEBUG MARKER: oleg234-server.virtnet: executing load_modules_local [ 4489.065354] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 4489.103251] alg: No test for adler32 (adler32-zlib) [ 4489.969742] Lustre: Lustre: Build Version: 2.17.50_65_g69bb343 [ 4490.060189] LNet: Added LNI 192.168.202.134@tcp [8/256/0/180] [ 4491.655223] Key type lgssc registered [ 4492.083907] Lustre: Echo OBD driver; http://www.lustre.org/ [ 4494.902791] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 4494.909585] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 4496.082643] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 4497.452885] Lustre: DEBUG MARKER: oleg234-server.virtnet: executing set_default_debug -1 all [ 4500.702475] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 4500.887776] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 4502.403602] Lustre: DEBUG MARKER: oleg234-server.virtnet: executing set_default_debug -1 all [ 4504.256402] Lustre: DEBUG MARKER: oleg234-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 4505.435105] Lustre: DEBUG MARKER: oleg234-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 4507.848144] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 4508.011461] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 4509.970818] Lustre: DEBUG MARKER: oleg234-server.virtnet: executing set_default_debug -1 all [ 4511.752086] Lustre: DEBUG MARKER: oleg234-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 4520.300461] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:49665 to 0x280000401:49696) [ 4525.327628] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 4525.966723] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4525.972139] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 4525.975650] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4525.979935] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 4525.979963] LustreError: 218561:0:(osp_precreate.c:1263:osp_precreate_thread()) cfs_fail_timeout id 2107 sleeping for 20000ms [ 4527.584531] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4527.589324] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 4527.592459] Lustre: Skipped 1 previous similar message [ 4528.563757] Lustre: lustre-OST0000: Not available for connect from 192.168.202.34@tcp (stopping) [ 4532.705914] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 4532.708217] Lustre: Skipped 1 previous similar message [ 4532.719864] LustreError: 220961:0:(obd_class.h:479:obd_check_dev()) Device 21 not setup [ 4532.755232] Lustre: server umount lustre-OST0000 complete [ 4546.055121] LustreError: 218561:0:(osp_precreate.c:1263:osp_precreate_thread()) cfs_fail_timeout id 2107 awake [ 4560.885511] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 4561.003956] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 4562.849448] Lustre: DEBUG MARKER: oleg234-server.virtnet: executing set_default_debug -1 all [ 4564.605624] Lustre: DEBUG MARKER: oleg234-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 4569.218832] Lustre: DEBUG MARKER: osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid in FULL state after 4 sec [ 4570.413894] Lustre: DEBUG MARKER: oleg234-client.virtnet: executing wait_import_state (FULL|IDLE) osc.lustre-OST0000-osc-ffff8e8549b57000.ost_server_uuid 50 [ 4570.897697] Lustre: DEBUG MARKER: osc.lustre-OST0000-osc-ffff8e8549b57000.ost_server_uuid in FULL state after 0 sec [ 4577.962471] LustreError: 222278:0:(obd_class.h:479:obd_check_dev()) Device 21 not setup [ 4577.964928] LustreError: 222278:0:(obd_class.h:479:obd_check_dev()) Skipped 3 previous similar messages [ 4578.005640] Lustre: server umount lustre-OST0000 complete [ 4582.369710] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4582.371053] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 4582.377630] Lustre: Skipped 2 previous similar messages [ 4585.406863] LustreError: 222479:0:(obd_class.h:479:obd_check_dev()) Device 9 not setup [ 4585.409319] LustreError: 222479:0:(obd_class.h:479:obd_check_dev()) Skipped 3 previous similar messages [ 4585.446120] Lustre: server umount lustre-MDT0000 complete [ 4586.769164] LustreError: 218521:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1770689342 with bad export cookie 4280390456335151452 [ 4586.772204] LustreError: MGC192.168.202.134@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 4586.773456] LustreError: 218521:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) Skipped 3 previous similar messages [ 4586.899988] Lustre: server umount lustre-MDT0001 complete [ 4590.656739] Lustre: DEBUG MARKER: oleg234-server.virtnet: executing unload_modules_local [ 4591.681325] Key type lgssc unregistered [ 4591.799331] LNet: 223211:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 4591.801735] LNetError: 223211:0:(acceptor.c:264:lnet_acceptor_remove_socket()) Interface ens2 not found [ 4591.813341] LNet: Removed LNI 192.168.202.134@tcp [ 4592.121487] Key type .llcrypt unregistered [ 4592.122512] Key type ._llcrypt unregistered [ 4603.402973] Key type ._llcrypt registered [ 4603.404388] Key type .llcrypt registered [ 4603.437910] Lustre: DEBUG MARKER: oleg234-server.virtnet: executing load_modules_local [ 4603.743401] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 4603.768597] alg: No test for adler32 (adler32-zlib) [ 4604.617447] Lustre: Lustre: Build Version: 2.17.50_65_g69bb343 [ 4604.700826] LNet: Added LNI 192.168.202.134@tcp [8/256/0/180] [ 4606.279147] Key type lgssc registered [ 4606.649000] Lustre: Echo OBD driver; http://www.lustre.org/ [ 4609.877231] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 4609.882715] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 4611.041140] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 4612.327897] Lustre: DEBUG MARKER: oleg234-server.virtnet: executing set_default_debug -1 all [ 4615.289575] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 4615.421862] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 4616.781567] Lustre: DEBUG MARKER: oleg234-server.virtnet: executing set_default_debug -1 all [ 4617.755786] Lustre: Modifying parameter general.debug_raw_pointers=Y in log params [ 4619.903406] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 4620.035615] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 4621.095850] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:49698 to 0x280000401:49728) [ 4622.024259] Lustre: DEBUG MARKER: oleg234-server.virtnet: executing set_default_debug -1 all [ 4624.806851] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 4624.873516] Lustre: lustre-OST0001: new disk, initializing [ 4624.875365] Lustre: srv-lustre-OST0001: No data found on store. Initialize space. [ 4624.878260] Lustre: 227156:0:(osd_compat.c:1354:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 4624.908229] Lustre: lustre-OST0001: Imperative Recovery not enabled, recovery window 60-180 [ 4626.722715] Lustre: DEBUG MARKER: oleg234-server.virtnet: executing set_default_debug -1 all [ 4627.520765] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x00000002c0000400-0x0000000300000400]:1:ost [ 4627.523929] Lustre: cli-lustre-OST0001-super: Allocated super-sequence [0x00000002c0000400-0x0000000300000400]:1:ost] [ 4627.548977] Lustre: lustre-OST0001-osc-MDT0000: update sequence from 0x100010000 to 0x2c0000401 [ 4631.974780] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 4638.571647] Lustre: Setting parameter general.lod.*.mdt_hash=crush in log params [ 4651.488326] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 4651.491488] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4651.497637] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 4653.026819] Lustre: lustre-MDT0000-lwp-OST0000: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4653.026988] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 4653.030874] Lustre: Skipped 1 previous similar message [ 4653.032951] Lustre: Skipped 1 previous similar message [ 4656.972637] LustreError: 228246:0:(obd_class.h:479:obd_check_dev()) Device 28 not setup [ 4657.030648] Lustre: server umount lustre-MDT0000 complete [ 4658.145172] LustreError: 224896:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 4658.150808] LustreError: 224896:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 5 previous similar messages [ 4658.268062] LustreError: 224876:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1770689414 with bad export cookie 5280242642675872767 [ 4658.271920] LustreError: MGC192.168.202.134@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 4658.275176] LustreError: 224876:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 4658.339227] LustreError: 228449:0:(obd_class.h:479:obd_check_dev()) Device 27 not setup [ 4658.341898] LustreError: 228449:0:(obd_class.h:479:obd_check_dev()) Skipped 7 previous similar messages [ 4658.390020] Lustre: server umount lustre-MDT0001 complete [ 4669.441311] LustreError: 228649:0:(obd_class.h:479:obd_check_dev()) Device 21 not setup [ 4669.443752] LustreError: 228649:0:(obd_class.h:479:obd_check_dev()) Skipped 7 previous similar messages [ 4669.465227] Lustre: server umount lustre-OST0000 complete [ 4680.952100] LustreError: 228851:0:(obd_class.h:479:obd_check_dev()) Device 2 not setup [ 4680.956023] LustreError: 228851:0:(obd_class.h:479:obd_check_dev()) Skipped 3 previous similar messages [ 4681.001743] Lustre: server umount lustre-OST0001 complete [ 4687.733898] Lustre: DEBUG MARKER: oleg234-server.virtnet: executing load_modules_local [ 4691.220098] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 4691.386256] LustreError: 230229:0:(ldlm_lib.c:1179:target_handle_connect()) mylustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 4691.412138] Lustre: mylustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 4692.680209] Lustre: DEBUG MARKER: oleg234-server.virtnet: executing set_default_debug -1 all [ 4695.450631] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 4696.905514] Lustre: DEBUG MARKER: oleg234-server.virtnet: executing set_default_debug -1 all [ 4697.920561] Lustre: 231336:0:(mgs_llog.c:1348:mgs_modify_param()) MGS: modify general/debug_raw_pointers=Y (mode = 0) failed: rc = -17 [ 4700.257120] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 4700.395625] Lustre: mylustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 4700.398130] Lustre: Skipped 1 previous similar message [ 4701.414852] LustreError: 231691:0:(ldlm_lib.c:1179:target_handle_connect()) mylustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 4701.420297] Lustre: mylustre-OST0000: new connection from mylustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:49730 to 0x280000401:49760) [ 4702.380130] Lustre: DEBUG MARKER: oleg234-server.virtnet: executing set_default_debug -1 all [ 4705.204049] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 4706.284116] Lustre: mylustre-OST0001: new connection from mylustre-MDT0000-mdtlov (cleaning up unused objects from 0x2c0000401:3 to 0x2c0000401:33) [ 4707.060077] Lustre: DEBUG MARKER: oleg234-server.virtnet: executing set_default_debug -1 all [ 4710.125176] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 4711.354478] Lustre: 233176:0:(mgs_llog.c:1348:mgs_modify_param()) MGS: modify general/lod.*.mdt_hash=crush (mode = 0) failed: rc = -17 [ 4721.119925] LustreError: mylustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 4721.122703] Lustre: mylustre-MDT0000-osp-MDT0001: Connection to mylustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4721.127257] Lustre: Skipped 1 previous similar message [ 4721.129628] Lustre: mylustre-MDT0000: Not available for connect from 0@lo (stopping) [ 4726.753955] Lustre: mylustre-MDT0000: Not available for connect from 0@lo (stopping) [ 4726.756540] Lustre: Skipped 5 previous similar messages [ 4726.784585] LustreError: 233483:0:(obd_class.h:479:obd_check_dev()) Device 10 not setup [ 4726.786655] LustreError: 233483:0:(obd_class.h:479:obd_check_dev()) Skipped 5 previous similar messages [ 4726.823000] Lustre: server umount mylustre-MDT0000 complete [ 4728.040431] LustreError: 230205:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1770689483 with bad export cookie 5280242642675875553 [ 4728.043980] LustreError: 230205:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) Skipped 2 previous similar messages [ 4728.044360] LustreError: MGC192.168.202.134@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 4728.155868] Lustre: server umount mylustre-MDT0001 complete [ 4740.084101] LustreError: 233884:0:(obd_class.h:479:obd_check_dev()) Device 23 not setup [ 4740.085984] LustreError: 233884:0:(obd_class.h:479:obd_check_dev()) Skipped 15 previous similar messages [ 4740.103956] Lustre: server umount mylustre-OST0000 complete [ 4751.649102] Lustre: server umount mylustre-OST0001 complete [ 4758.177483] Lustre: DEBUG MARKER: oleg234-server.virtnet: executing load_modules_local [ 4761.778815] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 4762.040482] LustreError: 235464:0:(ldlm_lib.c:1179:target_handle_connect()) tfs-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 4762.066504] Lustre: tfs-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 4762.068253] Lustre: Skipped 1 previous similar message [ 4763.442147] Lustre: DEBUG MARKER: oleg234-server.virtnet: executing set_default_debug -1 all [ 4766.438081] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 4767.798935] Lustre: DEBUG MARKER: oleg234-server.virtnet: executing set_default_debug -1 all [ 4768.728277] Lustre: 236568:0:(mgs_llog.c:1348:mgs_modify_param()) MGS: modify general/debug_raw_pointers=Y (mode = 0) failed: rc = -17 [ 4770.811624] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 4772.002525] LustreError: 236923:0:(ldlm_lib.c:1179:target_handle_connect()) tfs-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 4772.006746] Lustre: tfs-OST0000: new connection from tfs-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:49762 to 0x280000401:49792) [ 4772.715592] Lustre: DEBUG MARKER: oleg234-server.virtnet: executing set_default_debug -1 all [ 4775.430186] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 4776.555231] Lustre: tfs-OST0001: new connection from tfs-MDT0000-mdtlov (cleaning up unused objects from 0x2c0000401:3 to 0x2c0000401:65) [ 4777.455762] Lustre: DEBUG MARKER: oleg234-server.virtnet: executing set_default_debug -1 all [ 4780.511132] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 4781.659151] Lustre: 238412:0:(mgs_llog.c:1348:mgs_modify_param()) MGS: modify general/lod.*.mdt_hash=crush (mode = 0) failed: rc = -17 [ 4792.292979] Lustre: tfs-MDT0000-lwp-OST0001: Connection to tfs-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4792.296721] Lustre: tfs-MDT0000: Not available for connect from 0@lo (stopping) [ 4792.297490] Lustre: Skipped 6 previous similar messages [ 4792.299237] Lustre: Skipped 2 previous similar messages [ 4792.301257] LustreError: tfs-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 4794.750380] LustreError: 238663:0:(obd_class.h:479:obd_check_dev()) Device 10 not setup [ 4794.752609] LustreError: 238663:0:(obd_class.h:479:obd_check_dev()) Skipped 9 previous similar messages [ 4794.780147] Lustre: server umount tfs-MDT0000 complete [ 4796.212090] LustreError: 238415:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1770689551 with bad export cookie 5280242642675877450 [ 4796.212613] LustreError: MGC192.168.202.134@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 4796.216022] LustreError: 238415:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) Skipped 1 previous similar message [ 4796.375372] Lustre: server umount tfs-MDT0001 complete [ 4808.065713] Lustre: server umount tfs-OST0000 complete [ 4819.674860] Lustre: server umount tfs-OST0001 complete [ 4826.301891] Lustre: DEBUG MARKER: oleg234-server.virtnet: executing load_modules_local [ 4829.587449] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 4829.751324] LustreError: 240646:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 4829.773877] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 4829.776364] Lustre: Skipped 3 previous similar messages [ 4831.041689] Lustre: DEBUG MARKER: oleg234-server.virtnet: executing set_default_debug -1 all [ 4833.791771] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 4835.208205] Lustre: DEBUG MARKER: oleg234-server.virtnet: executing set_default_debug -1 all [ 4836.198230] Lustre: 241754:0:(mgs_llog.c:1348:mgs_modify_param()) MGS: modify general/debug_raw_pointers=Y (mode = 0) failed: rc = -17 [ 4838.394041] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 4839.526569] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:49762 to 0x280000401:49824) [ 4840.405471] Lustre: DEBUG MARKER: oleg234-server.virtnet: executing set_default_debug -1 all [ 4843.210154] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 4844.328094] Lustre: lustre-OST0001: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x2c0000401:3 to 0x2c0000401:97) [ 4845.076537] Lustre: DEBUG MARKER: oleg234-server.virtnet: executing set_default_debug -1 all [ 4848.104224] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 4849.249653] Lustre: 243595:0:(mgs_llog.c:1348:mgs_modify_param()) MGS: modify general/lod.*.mdt_hash=crush (mode = 0) failed: rc = -17 [ 4856.333609] Lustre: DEBUG MARKER: == conf-sanity test 104a: Make sure user defined options are reflected in mount ========================================================== 21:13:31 (1770689611) [ 4870.118411] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4870.120973] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 4870.122570] Lustre: Skipped 3 previous similar messages [ 4870.125471] Lustre: Skipped 2 previous similar messages [ 4870.127666] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 4874.245354] LustreError: 243964:0:(obd_class.h:479:obd_check_dev()) Device 10 not setup [ 4874.248350] LustreError: 243964:0:(obd_class.h:479:obd_check_dev()) Skipped 25 previous similar messages [ 4874.285127] Lustre: server umount lustre-MDT0000 complete [ 4875.237307] LustreError: 240641:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 4875.243087] LustreError: 240641:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 4 previous similar messages [ 4875.573144] LustreError: 240620:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1770689631 with bad export cookie 5280242642675879123 [ 4875.576309] LustreError: 240620:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) Skipped 2 previous similar messages [ 4875.576845] LustreError: MGC192.168.202.134@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 4880.353270] Lustre: lustre-MDT0001-lwp-OST0001: Connection to lustre-MDT0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4880.362346] Lustre: Skipped 1 previous similar message [ 4881.787739] Lustre: server umount lustre-MDT0001 complete [ 4889.279856] Lustre: server umount lustre-OST0000 complete [ 4896.728090] Lustre: server umount lustre-OST0001 complete [ 4898.701877] Lustre: DEBUG MARKER: oleg234-server.virtnet: executing set_hostid [ 4901.695772] Lustre: DEBUG MARKER: oleg234-server.virtnet: executing load_modules_local [ 4913.007861] LDISKFS-fs (dm-0): Mount option "noacl" will be removed by 3.5 [ 4913.007861] Contact linux-ldiskfs@vger.kernel.org if you think we should keep it. [ 4913.007861] [ 4913.029877] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: acl,user_xattr,errors=remount-ro,noacl,no_mbcache,nodelalloc [ 4913.117554] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 4913.129711] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 4913.166566] Lustre: lustre-MDT0000: new disk, initializing [ 4913.189817] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 4913.192029] Lustre: Skipped 3 previous similar messages [ 4913.198117] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 4914.427643] Lustre: DEBUG MARKER: oleg234-server.virtnet: executing set_default_debug -1 all [ 4918.487328] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 4918.524501] Lustre: 246736:0:(mgs_llog.c:1348:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 4918.546261] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [ 4918.548240] Lustre: Skipped 1 previous similar message [ 4918.584937] Lustre: lustre-MDT0001: new disk, initializing [ 4918.627484] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [ 4918.631846] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 4919.843286] Lustre: DEBUG MARKER: oleg234-server.virtnet: executing set_default_debug -1 all [ 4923.837403] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 4923.938986] Lustre: lustre-OST0000: new disk, initializing [ 4923.941266] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 4923.943744] Lustre: 248287:0:(osd_compat.c:1354:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 4925.781762] Lustre: DEBUG MARKER: oleg234-server.virtnet: executing set_default_debug -1 all [ 4925.944979] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [ 4925.947924] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 4925.982931] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 4929.820631] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 4929.879355] Lustre: lustre-OST0001: new disk, initializing [ 4929.881424] Lustre: srv-lustre-OST0001: No data found on store. Initialize space. [ 4929.883954] Lustre: 249141:0:(osd_compat.c:1354:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 4931.395748] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x00000002c0000400-0x0000000300000400]:1:ost [ 4931.399623] Lustre: cli-lustre-OST0001-super: Allocated super-sequence [0x00000002c0000400-0x0000000300000400]:1:ost] [ 4931.440444] Lustre: lustre-OST0001-osc-MDT0000: update sequence from 0x100010000 to 0x2c0000401 [ 4931.797887] Lustre: DEBUG MARKER: oleg234-server.virtnet: executing set_default_debug -1 all [ 4935.886631] Lustre: DEBUG MARKER: == conf-sanity test 104b: Mount uses last flock argument ========================================================== 21:14:50 (1770689690) [ 4939.329182] Lustre: DEBUG MARKER: == conf-sanity test 105: check file creation for ro and rw bind mnt pt ========================================================== 21:14:54 (1770689694) [ 4951.520454] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 4951.520600] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4951.523570] LustreError: Skipped 1 previous similar message [ 4951.529693] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 4951.533418] Lustre: Skipped 3 previous similar messages [ 4956.781615] LustreError: 250148:0:(obd_class.h:479:obd_check_dev()) Device 19 not setup [ 4956.783555] LustreError: 250148:0:(obd_class.h:479:obd_check_dev()) Skipped 25 previous similar messages [ 4956.801976] Lustre: server umount lustre-OST0000 complete [ 4957.156825] LustreError: 248656:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-OST0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 4957.160706] LustreError: 248656:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 3 previous similar messages [ 4959.711949] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 4964.258549] Lustre: server umount lustre-MDT0000 complete [ 4965.514369] LustreError: 246727:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1770689721 with bad export cookie 5280242642675880677 [ 4965.519803] LustreError: MGC192.168.202.134@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 4971.691725] Lustre: server umount lustre-MDT0001 complete [ 4975.054139] Lustre: DEBUG MARKER: oleg234-server.virtnet: executing unload_modules_local [ 4984.604042] Lustre: server umount lustre-OST0001 complete [ 4986.691031] Lustre: DEBUG MARKER: oleg234-server.virtnet: executing set_hostid [ 4992.655660] Lustre: DEBUG MARKER: oleg234-server.virtnet: executing load_modules_local [ 5004.488817] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 5004.584987] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 5004.599206] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 5004.641355] Lustre: lustre-MDT0000: new disk, initializing [ 5004.670282] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 5005.958165] Lustre: DEBUG MARKER: oleg234-server.virtnet: executing set_default_debug -1 all [ 5010.046677] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 5010.085060] Lustre: 253596:0:(mgs_llog.c:1348:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 5010.190908] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 5011.372221] Lustre: DEBUG MARKER: oleg234-server.virtnet: executing set_default_debug -1 all [ 5014.377273] Lustre: DEBUG MARKER: oleg234-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 5015.510920] Lustre: DEBUG MARKER: oleg234-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 5017.694327] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 5017.800765] Lustre: 255337:0:(osd_compat.c:1354:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 5019.256206] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 5019.292384] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 5019.585959] Lustre: DEBUG MARKER: oleg234-server.virtnet: executing set_default_debug -1 all [ 5022.548231] Lustre: DEBUG MARKER: oleg234-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 5029.857242] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 5029.858789] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5029.860861] LustreError: Skipped 1 previous similar message [ 5029.864942] Lustre: Skipped 5 previous similar messages [ 5029.866813] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 5029.870046] Lustre: Skipped 9 previous similar messages [ 5031.002991] Lustre: server umount lustre-OST0000 complete [ 5038.366626] Lustre: server umount lustre-MDT0000 complete [ 5039.578189] LustreError: 254375:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1770689795 with bad export cookie 5280242642675882147 [ 5039.581539] LustreError: MGC192.168.202.134@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 5039.582215] LustreError: 254375:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) Skipped 8 previous similar messages [ 5039.701884] Lustre: server umount lustre-MDT0001 complete [ 5043.059808] Lustre: DEBUG MARKER: oleg234-server.virtnet: executing unload_modules_local [ 5044.018346] Key type lgssc unregistered [ 5044.135400] LNet: 256960:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 5044.137539] LNetError: 256960:0:(acceptor.c:264:lnet_acceptor_remove_socket()) Interface ens2 not found [ 5044.148378] LNet: Removed LNI 192.168.202.134@tcp [ 5044.452117] Key type .llcrypt unregistered [ 5044.453068] Key type ._llcrypt unregistered [ 5053.758986] Key type ._llcrypt registered [ 5053.760243] Key type .llcrypt registered [ 5053.796565] Lustre: DEBUG MARKER: oleg234-server.virtnet: executing load_modules_local [ 5054.105848] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 5054.141543] alg: No test for adler32 (adler32-zlib) [ 5055.002322] Lustre: Lustre: Build Version: 2.17.50_65_g69bb343 [ 5055.090201] LNet: Added LNI 192.168.202.134@tcp [8/256/0/180] [ 5056.671211] Key type lgssc registered [ 5057.001309] Lustre: Echo OBD driver; http://www.lustre.org/ [ 5059.436217] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 5059.441458] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 5060.597762] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 5061.800879] Lustre: DEBUG MARKER: oleg234-server.virtnet: executing set_default_debug -1 all [ 5064.495168] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 5064.619771] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 5065.872076] Lustre: DEBUG MARKER: oleg234-server.virtnet: executing set_default_debug -1 all [ 5067.505811] Lustre: DEBUG MARKER: oleg234-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 5068.581481] Lustre: DEBUG MARKER: oleg234-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 5070.638167] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 5070.747356] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 5071.781846] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:3 to 0x280000401:33) [ 5072.467864] Lustre: DEBUG MARKER: oleg234-server.virtnet: executing set_default_debug -1 all [ 5074.189150] Lustre: DEBUG MARKER: oleg234-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 5074.542439] Lustre: Setting parameter lustre-OST0000.ost.unknown_param=50 in log lustre-OST0000 [ 5076.959973] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 5076.962513] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5076.967300] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 5080.034376] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 5080.034920] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5080.035592] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 5081.259124] LustreError: 260667:0:(obd_class.h:479:obd_check_dev()) Device 21 not setup [ 5081.289757] Lustre: server umount lustre-OST0000 complete [ 5085.153084] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5085.153235] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 5085.157037] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 5085.162096] Lustre: Skipped 2 previous similar messages [ 5088.701341] LustreError: 260874:0:(obd_class.h:479:obd_check_dev()) Device 9 not setup [ 5088.703696] LustreError: 260874:0:(obd_class.h:479:obd_check_dev()) Skipped 3 previous similar messages [ 5088.724392] Lustre: server umount lustre-MDT0000 complete [ 5089.864972] LustreError: 258359:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1770689845 with bad export cookie 7622482600828954954 [ 5089.867541] LustreError: 258359:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) Skipped 2 previous similar messages [ 5089.868031] LustreError: MGC192.168.202.134@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 5089.930564] LustreError: 261075:0:(obd_class.h:479:obd_check_dev()) Device 16 not setup [ 5089.932318] LustreError: 261075:0:(obd_class.h:479:obd_check_dev()) Skipped 5 previous similar messages [ 5089.979765] Lustre: server umount lustre-MDT0001 complete [ 5093.312188] Lustre: DEBUG MARKER: oleg234-server.virtnet: executing unload_modules_local [ 5094.268156] Key type lgssc unregistered [ 5094.374357] LNet: 261605:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 5094.376974] LNetError: 261605:0:(acceptor.c:264:lnet_acceptor_remove_socket()) Interface ens2 not found [ 5094.383291] LNet: Removed LNI 192.168.202.134@tcp [ 5094.674830] Key type .llcrypt unregistered [ 5094.676268] Key type ._llcrypt unregistered [ 5101.141343] Key type ._llcrypt registered [ 5101.142350] Key type .llcrypt registered [ 5101.186298] Lustre: DEBUG MARKER: oleg234-server.virtnet: executing load_modules_local [ 5101.525221] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 5101.531614] alg: No test for adler32 (adler32-zlib) [ 5102.395969] Lustre: Lustre: Build Version: 2.17.50_65_g69bb343 [ 5102.488296] LNet: Added LNI 192.168.202.134@tcp [8/256/0/180] [ 5104.071167] Key type lgssc registered [ 5104.437692] Lustre: Echo OBD driver; http://www.lustre.org/ [ 5107.584428] Lustre: lustre-OST0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 5107.592157] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 5123.039386] LustreError: Failed to get MGS log lustre-sptlrpc and no local copy. [ 5128.159231] LustreError: 262760:0:(mgc_request.c:233:do_config_log_add()) MGC192.168.202.134@tcp: failed processing log, type 4: rc = -110 [ 5154.783172] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 5156.660097] Lustre: DEBUG MARKER: oleg234-server.virtnet: executing set_default_debug -1 all [ 5158.314628] Lustre: DEBUG MARKER: oleg234-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 5158.991210] LustreError: 263424:0:(obd_class.h:479:obd_check_dev()) Device 1 not setup [ 5159.026275] Lustre: server umount lustre-OST0000 complete [ 5164.101826] Lustre: DEBUG MARKER: oleg234-server.virtnet: executing unload_modules_local [ 5165.104506] Key type lgssc unregistered [ 5165.227335] LNet: 264259:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 5165.230268] LNetError: 264259:0:(acceptor.c:264:lnet_acceptor_remove_socket()) Interface ens2 not found [ 5165.243251] LNet: Removed LNI 192.168.202.134@tcp [ 5165.505173] Key type .llcrypt unregistered [ 5165.506286] Key type ._llcrypt unregistered [ 5178.238361] Key type ._llcrypt registered [ 5178.239301] Key type .llcrypt registered [ 5178.270472] Lustre: DEBUG MARKER: oleg234-server.virtnet: executing load_modules_local [ 5178.548970] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 5178.598094] alg: No test for adler32 (adler32-zlib) [ 5179.454711] Lustre: Lustre: Build Version: 2.17.50_65_g69bb343 [ 5179.542098] LNet: Added LNI 192.168.202.134@tcp [8/256/0/180] [ 5181.127167] Key type lgssc registered [ 5181.477645] Lustre: Echo OBD driver; http://www.lustre.org/ [ 5185.601449] loop0: detected capacity change from 0 to 204800000 [ 5185.655399] blk_update_request: operation not supported error, dev loop0, sector 399872 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [ 5185.658631] blk_update_request: operation not supported error, dev loop0, sector 208 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [ 5185.790148] blk_update_request: operation not supported error, dev loop0, sector 160232 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [ 5186.166850] loop0: detected capacity change from 0 to 204800000 [ 5186.229578] blk_update_request: operation not supported error, dev loop0, sector 399872 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [ 5186.233355] blk_update_request: operation not supported error, dev loop0, sector 208 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [ 5186.354522] blk_update_request: operation not supported error, dev loop0, sector 160232 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [ 5186.687971] loop0: detected capacity change from 0 to 204800000 [ 5186.742934] blk_update_request: operation not supported error, dev loop0, sector 399872 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [ 5186.747067] blk_update_request: operation not supported error, dev loop0, sector 8224 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [ 5186.786746] blk_update_request: operation not supported error, dev loop0, sector 58264 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [ 5187.143327] loop0: detected capacity change from 0 to 204800000 [ 5187.203177] blk_update_request: operation not supported error, dev loop0, sector 399872 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [ 5187.606755] loop0: detected capacity change from 0 to 204800000 [ 5187.615293] LDISKFS-fs (loop0): mounted filesystem with ordered data mode. Opts: (null) [ 5188.878761] loop0: detected capacity change from 0 to 204800000 [ 5188.888611] LDISKFS-fs (loop0): mounted filesystem with ordered data mode. Opts: (null) [ 5190.142894] loop0: detected capacity change from 0 to 204800000 [ 5190.149983] LDISKFS-fs (loop0): mounted filesystem with ordered data mode. Opts: (null) [ 5191.388280] loop0: detected capacity change from 0 to 204800000 [ 5191.397032] LDISKFS-fs (loop0): mounted filesystem with ordered data mode. Opts: (null) [ 5192.657842] loop0: detected capacity change from 0 to 204800000 [ 5192.681833] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 5192.692339] LDISKFS-fs (loop0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 5192.722190] Lustre: 267626:0:(osd_scrub.c:1842:osd_index_restore()) lustre-MDT0000: restore index 'fld' with [0x200000001:0x3:0x0]: rc = 0 [ 5192.725652] Lustre: 267626:0:(osd_scrub.c:1842:osd_index_restore()) lustre-MDT0000: restore index 'nodemap' with [0x200000003:0x3:0x0]: rc = 0 [ 5192.728630] Lustre: 267626:0:(osd_scrub.c:1842:osd_index_restore()) lustre-MDT0000: restore index '0x2010000-MDT0000' with [0x200000005:0x3:0x0]: rc = 0 [ 5192.732043] Lustre: 267626:0:(osd_scrub.c:1842:osd_index_restore()) lustre-MDT0000: restore index '0x2010000' with [0x200000003:0xc:0x0]: rc = 0 [ 5192.735407] Lustre: 267626:0:(osd_scrub.c:1842:osd_index_restore()) lustre-MDT0000: restore index '0x10000' with [0x200000003:0xa:0x0]: rc = 0 [ 5192.738598] Lustre: 267626:0:(osd_scrub.c:1842:osd_index_restore()) lustre-MDT0000: restore index '0x1010000' with [0x200000003:0xb:0x0]: rc = 0 [ 5192.741691] Lustre: 267626:0:(osd_scrub.c:1842:osd_index_restore()) lustre-MDT0000: restore index '0x1010000-MDT0000' with [0x200000005:0x2:0x0]: rc = 0 [ 5192.745204] Lustre: 267626:0:(osd_scrub.c:1842:osd_index_restore()) lustre-MDT0000: restore index '0x10000-MDT0000' with [0x200000005:0x1:0x0]: rc = 0 [ 5192.749112] Lustre: 267626:0:(osd_scrub.c:1842:osd_index_restore()) lustre-MDT0000: restore index '0x20000' with [0x200000006:0x20000:0x0]: rc = 0 [ 5192.752368] Lustre: 267626:0:(osd_scrub.c:1842:osd_index_restore()) lustre-MDT0000: restore index '0x2020000' with [0x200000006:0x2020000:0x0]: rc = 0 [ 5192.755702] Lustre: 267626:0:(osd_scrub.c:1842:osd_index_restore()) lustre-MDT0000: restore index '0x1020000' with [0x200000006:0x1020000:0x0]: rc = 0 [ 5192.759073] Lustre: 267626:0:(osd_scrub.c:1842:osd_index_restore()) lustre-MDT0000: restore index '0x2010000' with [0x200000006:0x2010000:0x0]: rc = 0 [ 5192.762984] Lustre: 267626:0:(osd_scrub.c:1842:osd_index_restore()) lustre-MDT0000: restore index '0x10000' with [0x200000006:0x10000:0x0]: rc = 0 [ 5192.766651] Lustre: 267626:0:(osd_scrub.c:1842:osd_index_restore()) lustre-MDT0000: restore index '0x1010000' with [0x200000006:0x1010000:0x0]: rc = 0 [ 5195.205758] LustreError: 267890:0:(obd_class.h:479:obd_check_dev()) Device 2 not setup [ 5195.249872] Lustre: server umount MGS complete [ 5195.529967] loop0: detected capacity change from 0 to 204800000 [ 5195.556029] LDISKFS-fs (loop0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 5195.717349] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 5195.734853] Lustre: lustre-MDT0000: Aborting client recovery [ 5196.016536] loop1: detected capacity change from 0 to 204800000 [ 5196.073132] LDISKFS-fs (loop1): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 5196.092812] Lustre: 268059:0:(osd_scrub.c:1842:osd_index_restore()) lustre-MDT0001: restore index 'fld' with [0x200000001:0x3:0x0]: rc = 0 [ 5196.095869] Lustre: 268059:0:(osd_scrub.c:1842:osd_index_restore()) lustre-MDT0001: restore index '0x2010000' with [0x200000003:0x6:0x0]: rc = 0 [ 5196.098924] Lustre: 268059:0:(osd_scrub.c:1842:osd_index_restore()) lustre-MDT0001: restore index '0x10000' with [0x200000003:0x4:0x0]: rc = 0 [ 5196.101927] Lustre: 268059:0:(osd_scrub.c:1842:osd_index_restore()) lustre-MDT0001: restore index '0x1010000-MDT0001' with [0x200000005:0x2:0x0]: rc = 0 [ 5196.106151] Lustre: 268059:0:(osd_scrub.c:1842:osd_index_restore()) lustre-MDT0001: restore index '0x1010000' with [0x200000003:0x5:0x0]: rc = 0 [ 5196.110072] Lustre: 268059:0:(osd_scrub.c:1842:osd_index_restore()) lustre-MDT0001: restore index '0x10000-MDT0001' with [0x200000005:0x1:0x0]: rc = 0 [ 5196.113329] Lustre: 268059:0:(osd_scrub.c:1842:osd_index_restore()) lustre-MDT0001: restore index '0x2010000-MDT0001' with [0x200000005:0x3:0x0]: rc = 0 [ 5196.119937] Lustre: Found index 1 for lustre-MDT0001, updating log [ 5196.503280] loop2: detected capacity change from 0 to 204800000 [ 5196.535388] LDISKFS-fs (loop2): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 5196.559167] Lustre: 268141:0:(osd_scrub.c:1842:osd_index_restore()) lustre-OST0000: restore index 'fld' with [0x200000001:0x3:0x0]: rc = 0 [ 5196.562687] Lustre: 268141:0:(osd_scrub.c:1842:osd_index_restore()) lustre-OST0000: restore index '0x1020000' with [0x200000003:0x6:0x0]: rc = 0 [ 5196.566422] Lustre: 268141:0:(osd_scrub.c:1842:osd_index_restore()) lustre-OST0000: restore index '0x2020000-OST0000' with [0x200000005:0x3:0x0]: rc = 0 [ 5196.570604] Lustre: 268141:0:(osd_scrub.c:1842:osd_index_restore()) lustre-OST0000: restore index '0x20000-OST0000' with [0x200000005:0x1:0x0]: rc = 0 [ 5196.575566] Lustre: 268141:0:(osd_scrub.c:1842:osd_index_restore()) lustre-OST0000: restore index '0x2020000' with [0x200000003:0x7:0x0]: rc = 0 [ 5196.578620] Lustre: 268141:0:(osd_scrub.c:1842:osd_index_restore()) lustre-OST0000: restore index '0x1020000-OST0000' with [0x200000005:0x2:0x0]: rc = 0 [ 5196.582043] Lustre: 268141:0:(osd_scrub.c:1842:osd_index_restore()) lustre-OST0000: restore index '0x20000' with [0x200000003:0x5:0x0]: rc = 0 [ 5196.637059] Lustre: Found index 0 for lustre-OST0000, updating log [ 5196.648378] Lustre: 268158:0:(osd_compat.c:1354:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 5196.665441] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 5196.667444] Lustre: Skipped 1 previous similar message [ 5196.673148] Lustre: lustre-OST0000: Aborting recovery [ 5196.955681] loop3: detected capacity change from 0 to 204800000 [ 5196.989237] LDISKFS-fs (loop3): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 5197.015449] Lustre: 268223:0:(osd_scrub.c:1842:osd_index_restore()) lustre-OST0001: restore index 'fld' with [0x200000001:0x3:0x0]: rc = 0 [ 5197.020595] Lustre: 268223:0:(osd_scrub.c:1842:osd_index_restore()) lustre-OST0001: restore index '0x20000' with [0x200000003:0x5:0x0]: rc = 0 [ 5197.027018] Lustre: 268223:0:(osd_scrub.c:1842:osd_index_restore()) lustre-OST0001: restore index '0x2020000-OST0001' with [0x200000005:0x3:0x0]: rc = 0 [ 5197.032523] Lustre: 268223:0:(osd_scrub.c:1842:osd_index_restore()) lustre-OST0001: restore index '0x2020000' with [0x200000003:0x7:0x0]: rc = 0 [ 5197.036409] Lustre: 268223:0:(osd_scrub.c:1842:osd_index_restore()) lustre-OST0001: restore index '0x1020000-OST0001' with [0x200000005:0x2:0x0]: rc = 0 [ 5197.041644] Lustre: 268223:0:(osd_scrub.c:1842:osd_index_restore()) lustre-OST0001: restore index '0x1020000' with [0x200000003:0x6:0x0]: rc = 0 [ 5197.045909] Lustre: 268223:0:(osd_scrub.c:1842:osd_index_restore()) lustre-OST0001: restore index '0x20000-OST0001' with [0x200000005:0x1:0x0]: rc = 0 [ 5202.407258] LustreError: 268001:0:(lod_dev.c:510:lod_sub_recovery_thread()) lustre-MDT0001-osp-MDT0000: get update log duration 7, retries 0, failed: rc = -108 [ 5205.474992] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x0:6 to 0x0:33) [ 5205.475620] Lustre: lustre-OST0001: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x0:6 to 0x0:33) [ 5205.475854] Lustre: lustre-OST0001: new connection from lustre-MDT0001-mdtlov (cleaning up unused objects from 0x2c0000400:5 to 0x2c0000400:33) [ 5205.481189] Lustre: lustre-OST0000: new connection from lustre-MDT0001-mdtlov (cleaning up unused objects from 0x280000400:5 to 0x280000400:33) [ 5205.486459] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000bd0 [ 5205.493816] Lustre: lustre-OST0001-osc-MDT0000: update sequence from 0x100010000 to 0x2c0000bd0 [ 5207.447562] LustreError: 267969:0:(mdt_identity.c:115:mdt_identity_do_upcall()) lustre-MDT0000: error invoking upcall /root/Work/Lustre/R5/lustre-release/lustre/tests/../utils/l_getidentity lustre-MDT0000 0: rc -2; check /proc/fs/lustre/mdt/lustre-MDT0000/identity_upcall, time 286us: rc = -2 [ 5220.832435] Lustre: lustre-MDT0000-lwp-OST0000: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5220.833407] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 5220.836138] Lustre: Skipped 2 previous similar messages [ 5220.838059] Lustre: Skipped 1 previous similar message [ 5224.840170] LustreError: 268576:0:(obd_class.h:479:obd_check_dev()) Device 10 not setup [ 5224.842383] LustreError: 268576:0:(obd_class.h:479:obd_check_dev()) Skipped 1 previous similar message [ 5224.898158] Lustre: server umount lustre-MDT0000 complete [ 5225.218730] LustreError: 267950:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1770689980 with bad export cookie 2361426621086900458 [ 5225.220862] LustreError: MGC192.168.202.134@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 5225.222351] LustreError: 267950:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 5225.951899] LustreError: 268453:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 5225.952194] Lustre: lustre-MDT0001-lwp-OST0001: Connection to lustre-MDT0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5225.952685] Lustre: lustre-MDT0001: Not available for connect from 0@lo (stopping) [ 5225.952698] Lustre: Skipped 2 previous similar messages [ 5225.956850] LustreError: 268453:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 3 previous similar messages [ 5225.963161] Lustre: Skipped 2 previous similar messages [ 5231.072780] LustreError: 268453:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 5231.072930] Lustre: lustre-MDT0001: Not available for connect from 0@lo (stopping) [ 5231.081466] LustreError: 268453:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 1 previous similar message [ 5231.083768] Lustre: Skipped 2 previous similar messages [ 5231.427788] LustreError: 268628:0:(obd_class.h:479:obd_check_dev()) Device 18 not setup [ 5231.430492] LustreError: 268628:0:(obd_class.h:479:obd_check_dev()) Skipped 7 previous similar messages [ 5231.510201] Lustre: server umount lustre-MDT0001 complete [ 5237.931057] LustreError: 268679:0:(obd_class.h:479:obd_check_dev()) Device 23 not setup [ 5237.932975] LustreError: 268679:0:(obd_class.h:479:obd_check_dev()) Skipped 7 previous similar messages [ 5237.964902] Lustre: server umount lustre-OST0000 complete [ 5244.398231] LustreError: 268729:0:(obd_class.h:479:obd_check_dev()) Device 2 not setup [ 5244.400449] LustreError: 268729:0:(obd_class.h:479:obd_check_dev()) Skipped 3 previous similar messages [ 5244.463526] Lustre: server umount lustre-OST0001 complete [ 5247.162660] Lustre: DEBUG MARKER: == conf-sanity test 109a: test lctl clear_conf fsname ==== 21:20:02 (1770690002) [ 5256.603571] Lustre: DEBUG MARKER: oleg234-server.virtnet: executing set_hostid [ 5259.792724] Lustre: DEBUG MARKER: oleg234-server.virtnet: executing load_modules_local [ 5271.483349] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 5271.575462] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 5271.589435] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 5271.628035] Lustre: lustre-MDT0000: new disk, initializing [ 5271.654091] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 5271.656470] Lustre: Skipped 1 previous similar message [ 5271.662687] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 5272.965938] Lustre: DEBUG MARKER: oleg234-server.virtnet: executing set_default_debug -1 all [ 5276.910268] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 5276.953825] Lustre: 272260:0:(mgs_llog.c:1348:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 5276.976993] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [ 5276.978826] Lustre: Skipped 1 previous similar message [ 5277.032681] Lustre: lustre-MDT0001: new disk, initializing [ 5277.069832] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 5277.082751] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [ 5277.086968] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 5278.337348] Lustre: DEBUG MARKER: oleg234-server.virtnet: executing set_default_debug -1 all [ 5281.338352] Lustre: DEBUG MARKER: oleg234-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 5282.616372] Lustre: DEBUG MARKER: oleg234-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 5284.748290] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 5284.863077] Lustre: lustre-OST0000: new disk, initializing [ 5284.865203] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 5284.868746] Lustre: 274002:0:(osd_compat.c:1354:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 5284.871041] Lustre: 274002:0:(osd_compat.c:1354:osd_object_spec_find()) Skipped 1 previous similar message [ 5284.904588] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 5286.796994] Lustre: DEBUG MARKER: oleg234-server.virtnet: executing set_default_debug -1 all [ 5288.566623] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [ 5288.570825] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 5288.600996] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 5290.000301] Lustre: DEBUG MARKER: oleg234-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 5292.711143] Lustre: Setting parameter lustre-MDT0000.mdd.atime_diff=62 in log lustre-MDT0000 [ 5295.364626] Lustre: Modifying parameter lustre-MDT0000.mdd.atime_diff=63 in log lustre-MDT0000 [ 5305.179311] Lustre: Setting parameter lustre-client.llite.max_read_ahead_mb=32 in log lustre-client [ 5345.250191] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5345.252618] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 5345.253715] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 5345.254991] Lustre: Skipped 1 previous similar message [ 5350.371271] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 5350.373572] Lustre: Skipped 2 previous similar messages [ 5350.444066] LustreError: 275863:0:(obd_class.h:479:obd_check_dev()) Device 19 not setup [ 5350.446224] LustreError: 275863:0:(obd_class.h:479:obd_check_dev()) Skipped 5 previous similar messages [ 5350.486551] Lustre: server umount lustre-OST0000 complete [ 5353.951680] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 5353.953572] LustreError: Skipped 1 previous similar message [ 5353.954623] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5357.847498] Lustre: server umount lustre-MDT0000 complete [ 5359.048508] LustreError: 272251:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1770690114 with bad export cookie 2361426621086903174 [ 5359.051866] LustreError: MGC192.168.202.134@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 5359.052354] LustreError: 272251:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 5359.166346] Lustre: server umount lustre-MDT0001 complete [ 5362.083192] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 5363.376174] Lustre: DEBUG MARKER: oleg234-server.virtnet: executing set_default_debug -1 all [ 5365.120670] Lustre: server umount MGS complete [ 5368.941067] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 5369.091189] LustreError: 277827:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 5369.097539] LustreError: 277839:0:(qmt_pool.c:1523:qmt_pool_add_rem()) lustre-QMT0000: can't remove lustre-OST0000_UUID pool 'pool1': rc = -22 [ 5369.119206] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 5370.368462] Lustre: DEBUG MARKER: oleg234-server.virtnet: executing set_default_debug -1 all [ 5373.179095] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 5374.693834] Lustre: DEBUG MARKER: oleg234-server.virtnet: executing set_default_debug -1 all [ 5376.511759] Lustre: DEBUG MARKER: oleg234-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 5377.710554] Lustre: DEBUG MARKER: oleg234-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 5379.972428] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 5381.986294] Lustre: DEBUG MARKER: oleg234-server.virtnet: executing set_default_debug -1 all [ 5383.772170] Lustre: DEBUG MARKER: oleg234-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 5408.737728] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 5408.737838] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5408.739535] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 5408.739540] Lustre: Skipped 3 previous similar messages [ 5408.750503] Lustre: Skipped 2 previous similar messages [ 5411.246105] LustreError: 280502:0:(obd_class.h:479:obd_check_dev()) Device 21 not setup [ 5411.248369] LustreError: 280502:0:(obd_class.h:479:obd_check_dev()) Skipped 19 previous similar messages [ 5411.289331] Lustre: server umount lustre-OST0000 complete [ 5418.709372] Lustre: server umount lustre-MDT0000 complete [ 5418.975930] LustreError: 277845:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 5418.981165] LustreError: 277845:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 1 previous similar message [ 5419.921354] LustreError: 280239:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1770690175 with bad export cookie 2361426621086904987 [ 5419.924971] LustreError: 280239:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 5419.925069] LustreError: MGC192.168.202.134@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 5420.029676] Lustre: server umount lustre-MDT0001 complete [ 5423.184837] Lustre: DEBUG MARKER: oleg234-server.virtnet: executing unload_modules_local [ 5424.159125] Key type lgssc unregistered [ 5424.279358] LNet: 281434:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 5424.282307] LNetError: 281434:0:(acceptor.c:264:lnet_acceptor_remove_socket()) Interface ens2 not found [ 5424.294266] LNet: Removed LNI 192.168.202.134@tcp [ 5424.574458] Key type .llcrypt unregistered [ 5424.575591] Key type ._llcrypt unregistered [ 5436.115518] Key type ._llcrypt registered [ 5436.116663] Key type .llcrypt registered [ 5436.151652] Lustre: DEBUG MARKER: oleg234-server.virtnet: executing set_hostid [ 5442.085767] Lustre: DEBUG MARKER: oleg234-server.virtnet: executing load_modules_local [ 5442.505670] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 5442.559389] alg: No test for adler32 (adler32-zlib) [ 5443.407552] Lustre: Lustre: Build Version: 2.17.50_65_g69bb343 [ 5443.484682] LNet: Added LNI 192.168.202.134@tcp [8/256/0/180] [ 5445.063148] Key type lgssc registered [ 5445.434513] Lustre: Echo OBD driver; http://www.lustre.org/ [ 5456.086057] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 5456.091343] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 5457.185857] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 5457.196243] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 5457.232365] Lustre: lustre-MDT0000: new disk, initializing [ 5457.254819] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 5457.261993] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 5458.453140] Lustre: DEBUG MARKER: oleg234-server.virtnet: executing set_default_debug -1 all [ 5462.415642] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 5462.458496] Lustre: 285325:0:(mgs_llog.c:1348:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 5462.480800] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [ 5462.482682] Lustre: Skipped 1 previous similar message [ 5462.516810] Lustre: lustre-MDT0001: new disk, initializing [ 5462.544857] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 5462.557297] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [ 5462.560174] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 5463.715468] Lustre: DEBUG MARKER: oleg234-server.virtnet: executing set_default_debug -1 all [ 5466.643213] Lustre: DEBUG MARKER: oleg234-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 5467.758243] Lustre: DEBUG MARKER: oleg234-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 5469.826072] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 5469.929469] Lustre: lustre-OST0000: new disk, initializing [ 5469.931320] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 5469.934866] Lustre: 287068:0:(osd_compat.c:1354:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 5469.961220] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 5471.739144] Lustre: DEBUG MARKER: oleg234-server.virtnet: executing set_default_debug -1 all [ 5474.295497] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [ 5474.298414] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 5474.331895] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 5474.616728] Lustre: DEBUG MARKER: oleg234-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 5477.975726] Lustre: Setting parameter lustre-MDT0000.mdd.atime_diff=62 in log lustre-MDT0000 [ 5485.523991] Lustre: Modifying parameter lustre-MDT0000.mdd.atime_diff=63 in log lustre-MDT0000 [ 5493.850535] Lustre: Setting parameter lustre-client.llite.max_read_ahead_mb=32 in log lustre-client [ 5499.356453] Lustre: Modifying parameter lustre-client.llite.max_read_ahead_mb=64 in log lustre-client [ 5540.834292] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 5540.836218] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5540.838736] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 5540.843753] Lustre: Skipped 1 previous similar message [ 5543.150065] LustreError: 289078:0:(obd_class.h:479:obd_check_dev()) Device 19 not setup [ 5543.191679] Lustre: server umount lustre-OST0000 complete [ 5545.952863] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5545.953339] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 5545.957366] Lustre: Skipped 1 previous similar message [ 5545.960492] Lustre: Skipped 2 previous similar messages [ 5550.653253] LustreError: 289279:0:(obd_class.h:479:obd_check_dev()) Device 22 not setup [ 5550.656500] LustreError: 289279:0:(obd_class.h:479:obd_check_dev()) Skipped 3 previous similar messages [ 5550.683909] Lustre: server umount lustre-MDT0000 complete [ 5551.926299] LustreError: 285318:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1770690307 with bad export cookie 12169877997183976396 [ 5551.929755] LustreError: MGC192.168.202.134@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 5551.931432] LustreError: 285318:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 5551.996299] LustreError: 289480:0:(obd_class.h:479:obd_check_dev()) Device 21 not setup [ 5551.998581] LustreError: 289480:0:(obd_class.h:479:obd_check_dev()) Skipped 5 previous similar messages [ 5552.052465] Lustre: server umount lustre-MDT0001 complete [ 5555.020061] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 5556.441198] Lustre: DEBUG MARKER: oleg234-server.virtnet: executing set_default_debug -1 all [ 5558.088856] LustreError: 290419:0:(obd_class.h:479:obd_check_dev()) Device 2 not setup [ 5558.090721] LustreError: 290419:0:(obd_class.h:479:obd_check_dev()) Skipped 7 previous similar messages [ 5558.111543] Lustre: server umount MGS complete [ 5562.031717] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 5562.194893] LustreError: 291041:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 5562.202660] LustreError: 291041:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 3 previous similar messages [ 5562.203444] LustreError: 291053:0:(qmt_pool.c:1523:qmt_pool_add_rem()) lustre-QMT0000: can't remove lustre-OST0000_UUID pool 'pool1': rc = -22 [ 5562.234904] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 5563.587144] Lustre: DEBUG MARKER: oleg234-server.virtnet: executing set_default_debug -1 all [ 5566.556115] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 5566.672446] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 5567.944314] Lustre: DEBUG MARKER: oleg234-server.virtnet: executing set_default_debug -1 all [ 5569.810915] Lustre: DEBUG MARKER: oleg234-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 5571.132850] Lustre: DEBUG MARKER: oleg234-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 5573.523016] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 5575.666905] Lustre: DEBUG MARKER: oleg234-server.virtnet: executing set_default_debug -1 all [ 5577.502303] Lustre: DEBUG MARKER: oleg234-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 5607.393489] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 5607.394946] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5607.396168] LustreError: Skipped 1 previous similar message [ 5607.399921] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 5607.403174] Lustre: Skipped 1 previous similar message [ 5609.837090] LustreError: 293712:0:(obd_class.h:479:obd_check_dev()) Device 21 not setup [ 5609.839729] LustreError: 293712:0:(obd_class.h:479:obd_check_dev()) Skipped 1 previous similar message [ 5609.880376] Lustre: server umount lustre-OST0000 complete [ 5612.512761] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5612.513625] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 5612.516887] Lustre: Skipped 2 previous similar messages [ 5617.305258] Lustre: server umount lustre-MDT0000 complete [ 5617.632466] LustreError: 291040:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 5617.640701] LustreError: 291040:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 1 previous similar message [ 5618.589374] LustreError: 291020:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1770690374 with bad export cookie 12169877997183978335 [ 5618.593765] LustreError: MGC192.168.202.134@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 5618.595060] LustreError: 291020:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) Skipped 2 previous similar messages [ 5618.666764] LustreError: 294114:0:(obd_class.h:479:obd_check_dev()) Device 16 not setup [ 5618.669071] LustreError: 294114:0:(obd_class.h:479:obd_check_dev()) Skipped 9 previous similar messages [ 5618.727719] Lustre: server umount lustre-MDT0001 complete [ 5622.588869] Lustre: DEBUG MARKER: oleg234-server.virtnet: executing unload_modules_local [ 5623.646914] Key type lgssc unregistered [ 5623.784393] LNet: 294645:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 5623.787723] LNetError: 294645:0:(acceptor.c:264:lnet_acceptor_remove_socket()) Interface ens2 not found [ 5623.795338] LNet: Removed LNI 192.168.202.134@tcp [ 5624.106924] Key type .llcrypt unregistered [ 5624.108052] Key type ._llcrypt unregistered [ 5630.998216] Key type ._llcrypt registered [ 5630.999255] Key type .llcrypt registered [ 5631.034367] Lustre: DEBUG MARKER: oleg234-server.virtnet: executing set_hostid [ 5637.146159] Lustre: DEBUG MARKER: oleg234-server.virtnet: executing load_modules_local [ 5637.611846] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 5637.695756] alg: No test for adler32 (adler32-zlib) [ 5638.567293] Lustre: Lustre: Build Version: 2.17.50_65_g69bb343 [ 5638.661109] LNet: Added LNI 192.168.202.134@tcp [8/256/0/180] [ 5640.247232] Key type lgssc registered [ 5640.674539] Lustre: Echo OBD driver; http://www.lustre.org/ [ 5649.153599] Lustre: DEBUG MARKER: oleg234-server.virtnet: executing load_modules_local [ 5654.413913] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 5654.419835] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 5655.532451] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 5655.543981] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 5655.581988] Lustre: lustre-MDT0000: new disk, initializing [ 5655.609322] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 5655.616068] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 5657.064165] Lustre: DEBUG MARKER: oleg234-server.virtnet: executing set_default_debug -1 all [ 5661.308296] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 5661.420618] Lustre: lustre-OST0000: new disk, initializing [ 5661.422727] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 5661.424403] Lustre: Skipped 1 previous similar message [ 5661.428032] Lustre: 298595:0:(osd_compat.c:1354:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 5661.452451] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 5663.387837] Lustre: DEBUG MARKER: oleg234-server.virtnet: executing set_default_debug -1 all [ 5663.552267] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:0:ost [ 5663.556580] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:0:ost] [ 5663.567052] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x240000400 [ 6052.834783] Lustre: lustre-MDT0000-lwp-OST0000: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 6052.839894] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 6056.206515] LustreError: 299577:0:(obd_class.h:479:obd_check_dev()) Device 13 not setup [ 6056.300374] Lustre: server umount lustre-MDT0000 complete [ 6058.159499] LustreError: 298594:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1770690813 with bad export cookie 2280990939286725887 [ 6058.164743] LustreError: MGC192.168.202.134@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 6058.166645] LustreError: 298594:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) Skipped 2 previous similar messages [ 6058.194155] LustreError: 299779:0:(obd_class.h:479:obd_check_dev()) Device 2 not setup [ 6058.198214] LustreError: 299779:0:(obd_class.h:479:obd_check_dev()) Skipped 3 previous similar messages [ 6058.271808] Lustre: server umount lustre-OST0000 complete [ 6070.864370] Lustre: DEBUG MARKER: oleg234-server.virtnet: executing set_hostid [ 6075.721491] Lustre: DEBUG MARKER: oleg234-server.virtnet: executing load_modules_local [ 6093.590944] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 6093.719834] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 6093.746742] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 6093.791329] Lustre: lustre-MDT0000: new disk, initializing [ 6093.854569] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 6093.872609] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 6095.765306] Lustre: DEBUG MARKER: oleg234-server.virtnet: executing set_default_debug -1 all [ 6101.973773] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 6102.035365] Lustre: 302806:0:(mgs_llog.c:1348:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 6102.066066] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [ 6102.069852] Lustre: Skipped 1 previous similar message [ 6102.121664] Lustre: lustre-MDT0001: new disk, initializing [ 6102.185821] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 6102.214669] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [ 6102.224610] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 6104.246862] Lustre: DEBUG MARKER: oleg234-server.virtnet: executing set_default_debug -1 all [ 6108.590565] Lustre: DEBUG MARKER: oleg234-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 6110.283129] Lustre: DEBUG MARKER: oleg234-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 6113.590607] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 6113.706745] Lustre: lustre-OST0000: new disk, initializing [ 6113.709082] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 6113.714539] Lustre: 304551:0:(osd_compat.c:1354:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 6113.754838] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 6116.402356] Lustre: DEBUG MARKER: oleg234-server.virtnet: executing set_default_debug -1 all [ 6117.507768] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [ 6117.512515] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 6117.563176] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 6120.701620] Lustre: DEBUG MARKER: oleg234-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 6122.762299] Lustre: DEBUG MARKER: oleg234-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 [ 6122.857445] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [ 6124.775842] Lustre: DEBUG MARKER: oleg234-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid 50 [ 6124.889393] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [ 6128.096091] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 6128.100437] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 6128.108331] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 6128.110816] Lustre: Skipped 1 previous similar message [ 6133.218398] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 6133.220724] Lustre: Skipped 1 previous similar message [ 6138.339472] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 6138.342353] Lustre: Skipped 1 previous similar message [ 6139.871133] Lustre: lustre-OST0000 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 6139.895108] LustreError: 305741:0:(obd_class.h:479:obd_check_dev()) Device 19 not setup [ 6139.898608] LustreError: 305741:0:(obd_class.h:479:obd_check_dev()) Skipped 3 previous similar messages [ 6139.945860] Lustre: server umount lustre-OST0000 complete [ 6143.461111] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 6143.461938] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 6143.462601] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 6143.466128] Lustre: Skipped 2 previous similar messages [ 6143.469373] LustreError: Skipped 1 previous similar message [ 6147.330929] LustreError: 305944:0:(obd_class.h:479:obd_check_dev()) Device 22 not setup [ 6147.333478] LustreError: 305944:0:(obd_class.h:479:obd_check_dev()) Skipped 3 previous similar messages [ 6147.362203] Lustre: server umount lustre-MDT0000 complete [ 6149.132660] LustreError: 305945:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1770690904 with bad export cookie 2280990939289666664 [ 6149.136617] LustreError: MGC192.168.202.134@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 6149.138561] LustreError: 305945:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) Skipped 3 previous similar messages [ 6149.310111] Lustre: server umount lustre-MDT0001 complete [ 6150.937203] Lustre: DEBUG MARKER: SKIP: conf-sanity test_111 skipping SLOW test 111 [ 6151.673918] Lustre: DEBUG MARKER: == conf-sanity test 112a: mount OST with no_create option ========================================================== 21:35:06 (1770690906) [ 6154.267527] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 6154.469688] LustreError: 306718:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 6154.477198] LustreError: 306718:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 3 previous similar messages [ 6154.500295] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 6156.089918] Lustre: DEBUG MARKER: oleg234-server.virtnet: executing set_default_debug -1 all [ 6159.808932] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 6159.840176] LustreError: 306718:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 6161.575813] Lustre: DEBUG MARKER: oleg234-server.virtnet: executing set_default_debug -1 all [ 6164.064297] Lustre: DEBUG MARKER: oleg234-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 6165.833368] Lustre: DEBUG MARKER: oleg234-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 6168.948656] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 6171.606710] Lustre: DEBUG MARKER: oleg234-server.virtnet: executing set_default_debug -1 all [ 6174.079665] Lustre: DEBUG MARKER: oleg234-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 6176.942562] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 6177.015320] Lustre: lustre-OST0001: new disk, initializing [ 6177.017778] Lustre: srv-lustre-OST0001: No data found on store. Initialize space. [ 6177.021493] Lustre: 309215:0:(osd_compat.c:1354:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 6177.066277] Lustre: lustre-OST0001: Imperative Recovery not enabled, recovery window 60-180 [ 6177.068803] Lustre: Skipped 2 previous similar messages [ 6179.445571] Lustre: DEBUG MARKER: oleg234-server.virtnet: executing set_default_debug -1 all [ 6182.403724] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x00000002c0000400-0x0000000300000400]:1:ost [ 6182.407685] Lustre: cli-lustre-OST0001-super: Allocated super-sequence [0x00000002c0000400-0x0000000300000400]:1:ost] [ 6182.424583] LustreError: 309788:0:(osp_precreate.c:969:osp_precreate_cleanup_orphans()) lustre-OST0001-osc-MDT0001: cannot cleanup orphans: rc = -1 [ 6183.458085] LustreError: 309794:0:(osp_precreate.c:969:osp_precreate_cleanup_orphans()) lustre-OST0001-osc-MDT0000: cannot cleanup orphans: rc = -1 [ 6183.463014] LustreError: 309794:0:(osp_precreate.c:969:osp_precreate_cleanup_orphans()) Skipped 2 previous similar messages [ 6183.674759] Lustre: DEBUG MARKER: oleg234-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 [ 6183.759042] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [ 6184.483779] LustreError: 309788:0:(osp_precreate.c:969:osp_precreate_cleanup_orphans()) lustre-OST0001-osc-MDT0001: cannot cleanup orphans: rc = -1 [ 6184.488214] LustreError: 309788:0:(osp_precreate.c:969:osp_precreate_cleanup_orphans()) Skipped 1 previous similar message [ 6185.378737] Lustre: DEBUG MARKER: oleg234-client.virtnet: executing wait_import_state (FULL|IDLE) osc.lustre-OST0000-osc-ffff8e85486f4800.ost_server_uuid 50 [ 6186.008818] Lustre: DEBUG MARKER: osc.lustre-OST0000-osc-ffff8e85486f4800.ost_server_uuid in FULL state after 0 sec [ 6186.531080] LustreError: 309794:0:(osp_precreate.c:969:osp_precreate_cleanup_orphans()) lustre-OST0001-osc-MDT0000: cannot cleanup orphans: rc = -1 [ 6186.535705] LustreError: 309794:0:(osp_precreate.c:969:osp_precreate_cleanup_orphans()) Skipped 3 previous similar messages [ 6187.660874] Lustre: DEBUG MARKER: oleg234-client.virtnet: executing wait_import_state (FULL|IDLE) osc.lustre-OST0001-osc-ffff8e85486f4800.ost_server_uuid 50 [ 6188.303476] Lustre: DEBUG MARKER: osc.lustre-OST0001-osc-ffff8e85486f4800.ost_server_uuid in FULL state after 0 sec [ 6189.615224] Lustre: lustre-OST0001-osc-MDT0000: update sequence from 0x100010000 to 0x2c0000401 [ 6202.804722] Lustre: lustre-OST0001: Not available for connect from 192.168.202.34@tcp (stopping) [ 6202.808237] Lustre: Skipped 1 previous similar message [ 6202.851727] Lustre: lustre-OST0001-osc-MDT0000: Connection to lustre-OST0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 6206.380113] LustreError: 310519:0:(obd_class.h:479:obd_check_dev()) Device 25 not setup [ 6206.382652] LustreError: 310519:0:(obd_class.h:479:obd_check_dev()) Skipped 13 previous similar messages [ 6206.406282] Lustre: server umount lustre-OST0001 complete [ 6207.970622] LustreError: 308311:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 6207.978388] LustreError: 308311:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 1 previous similar message [ 6211.040236] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 6211.043431] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 6211.047979] Lustre: Skipped 1 previous similar message [ 6213.093293] LustreError: 308310:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 6213.100559] LustreError: 308310:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 1 previous similar message [ 6214.292035] Lustre: server umount lustre-OST0000 complete [ 6216.672307] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 6216.675946] LustreError: Skipped 1 previous similar message [ 6221.759894] LustreError: 310922:0:(obd_class.h:479:obd_check_dev()) Device 28 not setup [ 6221.762478] LustreError: 310922:0:(obd_class.h:479:obd_check_dev()) Skipped 7 previous similar messages [ 6221.794941] Lustre: server umount lustre-MDT0000 complete [ 6223.117653] LustreError: 306697:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1770690978 with bad export cookie 2280990939289667546 [ 6223.120311] LustreError: MGC192.168.202.134@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 6223.121734] LustreError: 306697:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 6223.253893] Lustre: server umount lustre-MDT0001 complete [ 6226.836357] Lustre: DEBUG MARKER: oleg234-server.virtnet: executing unload_modules_local [ 6227.837188] Key type lgssc unregistered [ 6227.969329] LNet: 311655:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 6227.971849] LNetError: 311655:0:(acceptor.c:264:lnet_acceptor_remove_socket()) Interface ens2 not found [ 6227.983332] LNet: Removed LNI 192.168.202.134@tcp [ 6228.293758] Key type .llcrypt unregistered [ 6228.295310] Key type ._llcrypt unregistered [ 6238.689416] Key type ._llcrypt registered [ 6238.690451] Key type .llcrypt registered [ 6238.733163] Lustre: DEBUG MARKER: oleg234-server.virtnet: executing load_modules_local [ 6239.070166] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 6239.081593] alg: No test for adler32 (adler32-zlib) [ 6239.947590] Lustre: Lustre: Build Version: 2.17.50_65_g69bb343 [ 6240.042568] LNet: Added LNI 192.168.202.134@tcp [8/256/0/180] [ 6241.623166] Key type lgssc registered [ 6242.007576] Lustre: Echo OBD driver; http://www.lustre.org/ [ 6244.786829] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 6244.793719] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 6245.994280] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 6247.556744] Lustre: DEBUG MARKER: oleg234-server.virtnet: executing set_default_debug -1 all [ 6249.670837] Lustre: DEBUG MARKER: oleg234-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 6252.439526] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 6252.598359] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 6254.268096] Lustre: DEBUG MARKER: oleg234-server.virtnet: executing set_default_debug -1 all [ 6257.924422] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 6258.068990] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 6260.235127] Lustre: DEBUG MARKER: oleg234-server.virtnet: executing set_default_debug -1 all [ 6262.308031] Lustre: DEBUG MARKER: oleg234-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 6263.205472] LustreError: 314582:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 6263.216103] LustreError: 314582:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 2 previous similar messages [ 6265.011812] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 6265.111662] Lustre: lustre-OST0001: Imperative Recovery not enabled, recovery window 60-180 [ 6267.490281] Lustre: DEBUG MARKER: oleg234-server.virtnet: executing set_default_debug -1 all [ 6269.614899] Lustre: DEBUG MARKER: oleg234-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0001-osc-[-0-9a-f]*.ost_server_uuid [ 6270.953685] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:36 to 0x280000401:65) [ 6270.959574] Lustre: lustre-OST0001: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x2c0000401:3 to 0x2c0000401:33) [ 6276.614667] Lustre: DEBUG MARKER: oleg234-server.virtnet: executing wait_import_state (FULL|IDLE) os[cp].lustre-OST0001-osc-MDT0001.ost_server_uuid 50 [ 6276.695860] Lustre: DEBUG MARKER: os[cp].lustre-OST0001-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [ 6362.079969] LustreError: lustre-OST0001-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 6362.083069] Lustre: lustre-OST0001-osc-MDT0001: Connection to lustre-OST0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 6362.088365] Lustre: lustre-OST0001: Not available for connect from 0@lo (stopping) [ 6363.105939] LustreError: lustre-OST0001-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 6363.109084] Lustre: lustre-OST0001: Not available for connect from 0@lo (stopping) [ 6363.110059] Lustre: lustre-OST0001-osc-MDT0000: Connection to lustre-OST0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 6365.489057] LustreError: 316446:0:(obd_class.h:479:obd_check_dev()) Device 27 not setup [ 6365.510177] Lustre: server umount lustre-OST0001 complete [ 6368.224980] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 6368.226557] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 6368.227068] LustreError: 315211:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 6368.227076] LustreError: 315211:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 1 previous similar message [ 6368.228796] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 6368.233512] Lustre: Skipped 1 previous similar message [ 6368.247586] Lustre: Skipped 2 previous similar messages [ 6373.346475] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 6373.346478] LustreError: 315214:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 6373.355914] Lustre: Skipped 1 previous similar message [ 6373.427097] LustreError: 316648:0:(obd_class.h:479:obd_check_dev()) Device 23 not setup [ 6373.430266] LustreError: 316648:0:(obd_class.h:479:obd_check_dev()) Skipped 3 previous similar messages [ 6373.477589] Lustre: server umount lustre-OST0000 complete [ 6375.391922] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 6375.395368] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 6378.464324] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 6378.467163] Lustre: Skipped 2 previous similar messages [ 6388.704920] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 6388.707885] Lustre: Skipped 3 previous similar messages [ 6389.215145] Lustre: lustre-MDT0000 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 6389.260565] LustreError: 316849:0:(obd_class.h:479:obd_check_dev()) Device 10 not setup [ 6389.263452] LustreError: 316849:0:(obd_class.h:479:obd_check_dev()) Skipped 3 previous similar messages [ 6389.295530] Lustre: server umount lustre-MDT0000 complete [ 6390.816332] LustreError: 313013:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1770691146 with bad export cookie 12325196561412552070 [ 6390.818225] LustreError: MGC192.168.202.134@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 6390.821799] LustreError: 313013:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) Skipped 2 previous similar messages [ 6390.994756] Lustre: server umount lustre-MDT0001 complete [ 6394.874106] Lustre: DEBUG MARKER: oleg234-server.virtnet: executing unload_modules_local [ 6395.968962] Key type lgssc unregistered [ 6396.102503] LNet: 317584:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 6396.106600] LNetError: 317584:0:(acceptor.c:264:lnet_acceptor_remove_socket()) Interface ens2 not found [ 6396.117027] LNet: Removed LNI 192.168.202.134@tcp [ 6396.463926] Key type .llcrypt unregistered [ 6396.465467] Key type ._llcrypt unregistered [ 6416.722539] Key type ._llcrypt registered [ 6416.723770] Key type .llcrypt registered [ 6416.768891] Lustre: DEBUG MARKER: oleg234-server.virtnet: executing load_modules_local [ 6417.204041] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 6417.234116] alg: No test for adler32 (adler32-zlib) [ 6418.110510] Lustre: Lustre: Build Version: 2.17.50_65_g69bb343 [ 6418.210799] LNet: Added LNI 192.168.202.134@tcp [8/256/0/180] [ 6419.799138] Key type lgssc registered [ 6420.255926] Lustre: Echo OBD driver; http://www.lustre.org/ [ 6423.742256] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 6423.749569] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 6424.952366] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 6426.384811] Lustre: DEBUG MARKER: oleg234-server.virtnet: executing set_default_debug -1 all [ 6429.449929] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 6429.605153] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 6431.118826] Lustre: DEBUG MARKER: oleg234-server.virtnet: executing set_default_debug -1 all [ 6432.177084] Lustre: Modifying parameter general.debug_raw_pointers=Y in log params [ 6434.621702] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 6434.721812] LustreError: 321287:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-OST0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 6434.728398] LustreError: 321287:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 3 previous similar messages [ 6434.789581] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 6436.922560] Lustre: DEBUG MARKER: oleg234-server.virtnet: executing set_default_debug -1 all [ 6439.909868] LustreError: 321297:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 6439.916586] LustreError: 321297:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 1 previous similar message [ 6439.925528] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:36 to 0x280000401:97) [ 6440.206646] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 6440.295708] Lustre: lustre-OST0001: Imperative Recovery not enabled, recovery window 60-180 [ 6442.451795] Lustre: DEBUG MARKER: oleg234-server.virtnet: executing set_default_debug -1 all [ 6445.546979] Lustre: lustre-OST0001: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x2c0000401:3 to 0x2c0000401:65) [ 6446.845970] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 6451.841699] Lustre: Setting parameter general.lod.*.mdt_hash=crush in log params [ 6460.384278] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 6460.388279] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 6460.394283] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 6460.897679] Lustre: lustre-MDT0000-lwp-OST0000: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 6460.899524] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 6460.903230] Lustre: Skipped 2 previous similar messages [ 6460.905133] Lustre: Skipped 1 previous similar message [ 6466.016341] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 6466.019209] Lustre: Skipped 4 previous similar messages [ 6466.498275] LustreError: 323235:0:(obd_class.h:479:obd_check_dev()) Device 10 not setup [ 6466.529532] Lustre: server umount lustre-MDT0000 complete [ 6467.795244] LustreError: 319809:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1770691223 with bad export cookie 12833579084806272600 [ 6467.799430] LustreError: MGC192.168.202.134@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 6467.799934] LustreError: 319809:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 6467.863913] LustreError: 323436:0:(obd_class.h:479:obd_check_dev()) Device 18 not setup [ 6467.865923] LustreError: 323436:0:(obd_class.h:479:obd_check_dev()) Skipped 7 previous similar messages [ 6467.911658] Lustre: server umount lustre-MDT0001 complete [ 6478.323122] LustreError: 323636:0:(obd_class.h:479:obd_check_dev()) Device 23 not setup [ 6478.325615] LustreError: 323636:0:(obd_class.h:479:obd_check_dev()) Skipped 7 previous similar messages [ 6478.342519] Lustre: server umount lustre-OST0000 complete [ 6489.841130] LustreError: 323838:0:(obd_class.h:479:obd_check_dev()) Device 2 not setup [ 6489.843359] LustreError: 323838:0:(obd_class.h:479:obd_check_dev()) Skipped 3 previous similar messages [ 6489.885069] Lustre: server umount lustre-OST0001 complete [ 6495.139292] Lustre: DEBUG MARKER: oleg234-server.virtnet: executing load_modules_local [ 6498.891530] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 6499.074528] LustreError: 325014:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 6499.099265] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 6500.442478] Lustre: DEBUG MARKER: oleg234-server.virtnet: executing set_default_debug -1 all [ 6503.430877] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 6504.883382] Lustre: DEBUG MARKER: oleg234-server.virtnet: executing set_default_debug -1 all [ 6505.904354] Lustre: 326118:0:(mgs_llog.c:1348:mgs_modify_param()) MGS: modify general/debug_raw_pointers=Y (mode = 0) failed: rc = -17 [ 6508.175343] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 6508.298579] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 6508.301341] Lustre: Skipped 1 previous similar message [ 6509.347395] LustreError: 326475:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 6509.351644] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:36 to 0x280000401:129) [ 6510.320803] Lustre: DEBUG MARKER: oleg234-server.virtnet: executing set_default_debug -1 all [ 6513.168174] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 6514.290214] Lustre: lustre-OST0001: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x2c0000401:3 to 0x2c0000401:97) [ 6515.171151] Lustre: DEBUG MARKER: oleg234-server.virtnet: executing set_default_debug -1 all [ 6518.429233] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 6519.646888] Lustre: 327962:0:(mgs_llog.c:1348:mgs_modify_param()) MGS: modify general/lod.*.mdt_hash=crush (mode = 0) failed: rc = -17 [ 6529.503951] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 6529.506671] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 6529.511815] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 6531.837830] LustreError: 328068:0:(obd_class.h:479:obd_check_dev()) Device 10 not setup [ 6531.840944] LustreError: 328068:0:(obd_class.h:479:obd_check_dev()) Skipped 5 previous similar messages [ 6531.874457] Lustre: server umount lustre-MDT0000 complete [ 6533.178178] LustreError: 327285:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1770691288 with bad export cookie 12833579084806274287 [ 6533.181628] LustreError: MGC192.168.202.134@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 6533.182688] LustreError: 327285:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 6533.304545] Lustre: server umount lustre-MDT0001 complete [ 6544.307127] LustreError: 328470:0:(obd_class.h:479:obd_check_dev()) Device 23 not setup [ 6544.309309] LustreError: 328470:0:(obd_class.h:479:obd_check_dev()) Skipped 15 previous similar messages [ 6544.329169] Lustre: server umount lustre-OST0000 complete [ 6555.933711] Lustre: server umount lustre-OST0001 complete [ 6557.181950] Lustre: DEBUG MARKER: SKIP: conf-sanity test_114 skipping SLOW test 114 [ 6557.688332] Lustre: DEBUG MARKER: SKIP: conf-sanity test_115 skipping excluded test 115 [ 6558.236538] Lustre: DEBUG MARKER: == conf-sanity test 116: big size MDT support ============ 21:41:53 (1770691313) [ 6565.357929] Lustre: DEBUG MARKER: oleg234-server.virtnet: executing load_modules_local [ 6568.414054] loop0: detected capacity change from 0 to 1099511627776 [ 6568.571092] SGI XFS with ACLs, security attributes, quota, no debug enabled [ 6568.573677] *** VALIDATE xfs *** [ 6568.578411] XFS (loop0): Mounting V5 Filesystem [ 6568.588911] XFS (loop0): Ending clean mount [ 6568.999242] loop1: detected capacity change from 0 to 18691697672192 [ 6589.708886] Lustre: DEBUG MARKER: oleg234-server.virtnet: executing set_hostid [ 6593.000156] Lustre: DEBUG MARKER: oleg234-server.virtnet: executing load_modules_local [ 6605.645506] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 6605.751202] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 6605.765181] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 6605.808518] Lustre: lustre-MDT0000: new disk, initializing [ 6605.845656] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 6605.848443] Lustre: Skipped 1 previous similar message [ 6605.855964] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 6607.322739] Lustre: DEBUG MARKER: oleg234-server.virtnet: executing set_default_debug -1 all [ 6612.021275] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 6612.066490] Lustre: 333023:0:(mgs_llog.c:1348:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 6612.091651] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [ 6612.093733] Lustre: Skipped 1 previous similar message [ 6612.137715] Lustre: lustre-MDT0001: new disk, initializing [ 6612.211595] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [ 6612.216855] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 6613.701035] Lustre: DEBUG MARKER: oleg234-server.virtnet: executing set_default_debug -1 all [ 6616.983082] Lustre: DEBUG MARKER: oleg234-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 6618.249604] Lustre: DEBUG MARKER: oleg234-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 6620.702100] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 6620.835937] Lustre: lustre-OST0000: new disk, initializing [ 6620.838272] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 6620.841972] Lustre: 334768:0:(osd_compat.c:1354:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 6622.589628] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [ 6622.592522] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 6622.633111] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 6622.838560] Lustre: DEBUG MARKER: oleg234-server.virtnet: executing set_default_debug -1 all [ 6625.991764] Lustre: DEBUG MARKER: oleg234-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 6627.444432] Lustre: DEBUG MARKER: oleg234-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 [ 6627.522963] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [ 6628.943374] Lustre: DEBUG MARKER: oleg234-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid 50 [ 6629.033261] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [ 6632.930261] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 6632.932234] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 6632.933927] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 6632.938201] Lustre: Skipped 4 previous similar messages [ 6632.940563] Lustre: Skipped 4 previous similar messages [ 6643.169772] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 6643.173733] Lustre: Skipped 4 previous similar messages [ 6644.191172] Lustre: lustre-OST0000 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 6644.212265] LustreError: 335957:0:(obd_class.h:479:obd_check_dev()) Device 19 not setup [ 6644.218136] LustreError: 335957:0:(obd_class.h:479:obd_check_dev()) Skipped 9 previous similar messages [ 6644.272165] Lustre: server umount lustre-OST0000 complete [ 6648.290147] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 6648.290303] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 6648.293421] LustreError: Skipped 1 previous similar message [ 6648.299343] Lustre: Skipped 1 previous similar message [ 6651.744725] Lustre: server umount lustre-MDT0000 complete [ 6653.057324] LustreError: 333014:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1770691408 with bad export cookie 12833579084806275841 [ 6653.060641] LustreError: MGC192.168.202.134@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 6653.061528] LustreError: 333014:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 6653.186680] Lustre: server umount lustre-MDT0001 complete [ 6655.992564] XFS (loop0): Unmounting Filesystem [ 6657.582599] Lustre: DEBUG MARKER: == conf-sanity test 117: lctl get_param return errors properly ========================================================== 21:43:32 (1770691412) [ 6659.830196] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 6659.982171] LustreError: 337176:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 6660.005348] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 6660.007936] Lustre: Skipped 2 previous similar messages [ 6661.380668] Lustre: DEBUG MARKER: oleg234-server.virtnet: executing set_default_debug -1 all [ 6664.356148] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 6665.778549] Lustre: DEBUG MARKER: oleg234-server.virtnet: executing set_default_debug -1 all [ 6667.571689] Lustre: DEBUG MARKER: oleg234-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 6668.715077] Lustre: DEBUG MARKER: oleg234-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 6670.943347] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 6672.860987] Lustre: DEBUG MARKER: oleg234-server.virtnet: executing set_default_debug -1 all [ 6674.550418] Lustre: DEBUG MARKER: oleg234-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 6689.762570] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 6689.762794] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 6689.764108] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 6689.764114] Lustre: Skipped 2 previous similar messages [ 6689.766655] Lustre: Skipped 1 previous similar message [ 6690.092084] LustreError: 339535:0:(obd_class.h:479:obd_check_dev()) Device 21 not setup [ 6690.094503] LustreError: 339535:0:(obd_class.h:479:obd_check_dev()) Skipped 17 previous similar messages [ 6690.130060] Lustre: server umount lustre-OST0000 complete [ 6693.401210] Lustre: server umount lustre-MDT0000 complete [ 6694.585236] LustreError: 337157:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1770691450 with bad export cookie 12833579084806276723 [ 6694.589249] LustreError: 337157:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) Skipped 3 previous similar messages [ 6694.589870] LustreError: MGC192.168.202.134@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 6694.705594] Lustre: server umount lustre-MDT0001 complete [ 6698.237125] Lustre: DEBUG MARKER: oleg234-server.virtnet: executing unload_modules_local [ 6699.251978] Key type lgssc unregistered [ 6699.384419] LNet: 340469:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 6699.388138] LNetError: 340469:0:(acceptor.c:264:lnet_acceptor_remove_socket()) Interface ens2 not found [ 6699.394313] LNet: Removed LNI 192.168.202.134@tcp [ 6699.701117] Key type .llcrypt unregistered [ 6699.702250] Key type ._llcrypt unregistered [ 6709.433015] Key type ._llcrypt registered [ 6709.434068] Key type .llcrypt registered [ 6709.471282] Lustre: DEBUG MARKER: oleg234-server.virtnet: executing load_modules_local [ 6709.801129] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 6709.839630] alg: No test for adler32 (adler32-zlib) [ 6710.705289] Lustre: Lustre: Build Version: 2.17.50_65_g69bb343 [ 6710.795883] LNet: Added LNI 192.168.202.134@tcp [8/256/0/180] [ 6712.375129] Key type lgssc registered [ 6712.726380] Lustre: Echo OBD driver; http://www.lustre.org/ [ 6715.261152] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 6715.268156] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 6716.428353] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 6717.853866] Lustre: DEBUG MARKER: oleg234-server.virtnet: executing set_default_debug -1 all [ 6720.978064] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 6721.106365] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 6722.377752] Lustre: DEBUG MARKER: oleg234-server.virtnet: executing set_default_debug -1 all [ 6724.123678] Lustre: DEBUG MARKER: oleg234-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 6725.323782] Lustre: DEBUG MARKER: oleg234-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 6727.757533] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 6727.879976] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 6729.890861] Lustre: DEBUG MARKER: oleg234-server.virtnet: executing set_default_debug -1 all [ 6731.708657] Lustre: DEBUG MARKER: oleg234-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 6746.591903] LustreError: lustre-MDT0001-osp-MDT0000: operation mds_statfs to node 0@lo failed: rc = -107 [ 6746.594917] Lustre: lustre-MDT0001-osp-MDT0000: Connection to lustre-MDT0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 6746.601132] Lustre: lustre-MDT0001: Not available for connect from 0@lo (stopping) [ 6747.567889] Lustre: lustre-MDT0001: Not available for connect from 192.168.202.34@tcp (stopping) [ 6747.617635] Lustre: lustre-MDT0001-lwp-OST0000: Connection to lustre-MDT0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 6752.680957] Lustre: lustre-MDT0001: Not available for connect from 192.168.202.34@tcp (stopping) [ 6752.683751] Lustre: Skipped 2 previous similar messages [ 6757.801354] Lustre: lustre-MDT0001: Not available for connect from 192.168.202.34@tcp (stopping) [ 6757.805799] Lustre: Skipped 2 previous similar messages [ 6759.391111] Lustre: lustre-MDT0001 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 6759.422364] LustreError: 344303:0:(obd_class.h:479:obd_check_dev()) Device 16 not setup [ 6759.444185] Lustre: server umount lustre-MDT0001 complete [ 6762.888786] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 6762.919545] Lustre: MGS: Regenerating lustre-MDT0001 log by user request: rc = 0 [ 6762.921600] Lustre: Found index 1 for lustre-MDT0001, updating log [ 6762.922607] LustreError: 342019:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0001: not available for connect from 192.168.202.34@tcp (no target). If you are running an HA pair check that the target is mounted on the other server. [ 6762.928764] LustreError: 342019:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 2 previous similar messages [ 6762.934993] Lustre: 341982:0:(mgs_llog.c:1348:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 6762.976727] Lustre: lustre-MDT0001: Not available for connect from 0@lo (not set up) [ 6762.978978] Lustre: Skipped 3 previous similar messages [ 6763.044266] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 6764.315737] Lustre: DEBUG MARKER: oleg234-server.virtnet: executing set_default_debug -1 all [ 6768.101508] Lustre: lustre-MDT0001-lwp-OST0000: Connection restored to 0@lo (at 0@lo) [ 6770.149621] LustreError: 345424:0:(obd_mount.c:188:lustre_start_simple()) lustre-MDT0001-lwp-OST0000 attach error -17 [ 6770.151690] LustreError: 345424:0:(tgt_mount.c:669:lustre_lwp_setup()) lustre-MDT0001-lwp-OST0000: setup up failed: rc -17 [ 6770.154102] LustreError: 341986:0:(mgc_request.c:614:do_requeue()) failed processing log: -17 [ 6770.173906] LustreError: 345427:0:(obd_config.c:2001:class_config_llog_handler()) MGC192.168.202.134@tcp: cfg command failed: rc = -17 [ 6770.177188] Lustre: cmd=cf001 0:lustre-MDT0001-osp-MDT0000 1:osp 2:lustre-MDT0000-mdtlov_UUID [ 6770.177188] [ 7070.126316] Lustre: lustre-MDT0001: Not available for connect from 192.168.202.34@tcp (stopping) [ 7070.180707] Lustre: lustre-MDT0001-osp-MDT0000: Connection to lustre-MDT0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 7070.180886] LustreError: lustre-MDT0001-osp-MDT0000: operation mds_statfs to node 0@lo failed: rc = -107 [ 7073.021554] LustreError: 357016:0:(obd_class.h:479:obd_check_dev()) Device 16 not setup [ 7073.023482] LustreError: 357016:0:(obd_class.h:479:obd_check_dev()) Skipped 5 previous similar messages [ 7073.046892] Lustre: server umount lustre-MDT0001 complete [ 7075.240690] LustreError: 341989:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0001: not available for connect from 192.168.202.34@tcp (no target). If you are running an HA pair check that the target is mounted on the other server. [ 7076.458219] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 7076.487263] Lustre: MGS: Regenerating lustre-MDT0001 log by user request: rc = 0 [ 7076.488937] Lustre: Found index 1 for lustre-MDT0001, updating log [ 7076.500615] Lustre: 341981:0:(mgs_llog.c:1348:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 7076.611087] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 7077.893940] Lustre: DEBUG MARKER: oleg234-server.virtnet: executing set_default_debug -1 all [ 7081.957271] Lustre: lustre-MDT0001-lwp-OST0000: Connection restored to 0@lo (at 0@lo) [ 7084.518652] LustreError: 358184:0:(obd_mount.c:188:lustre_start_simple()) lustre-MDT0001-lwp-OST0000 attach error -17 [ 7084.521907] LustreError: 358184:0:(tgt_mount.c:669:lustre_lwp_setup()) lustre-MDT0001-lwp-OST0000: setup up failed: rc -17 [ 7084.525518] LustreError: 341986:0:(mgc_request.c:614:do_requeue()) failed processing log: -17 [ 7084.528742] LustreError: 341986:0:(mgc_request.c:614:do_requeue()) Skipped 1 previous similar message [ 7084.547975] LustreError: 358187:0:(obd_config.c:689:class_setup()) Device 8 already setup (type osp) [ 7084.550488] LustreError: 358187:0:(obd_config.c:797:class_detach()) OBD device 8 still set up [ 7084.552818] LustreError: 358187:0:(obd_config.c:2001:class_config_llog_handler()) MGC192.168.202.134@tcp: cfg command failed: rc = -17 [ 7084.556139] Lustre: cmd=cf003 0:lustre-MDT0001-osp-MDT0000 1:lustre-MDT0001_UUID 2:192.168.202.134@tcp [ 7084.556139] [ 7382.447517] Lustre: lustre-MDT0001: Not available for connect from 192.168.202.34@tcp (stopping) [ 7382.450179] Lustre: Skipped 2 previous similar messages [ 7384.033757] Lustre: lustre-MDT0001-lwp-OST0000: Connection to lustre-MDT0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 7384.034072] LustreError: lustre-MDT0001-osp-MDT0000: operation mds_statfs to node 0@lo failed: rc = -107 [ 7384.038370] Lustre: Skipped 1 previous similar message [ 7386.239161] LustreError: 369672:0:(obd_class.h:479:obd_check_dev()) Device 16 not setup [ 7386.242207] LustreError: 369672:0:(obd_class.h:479:obd_check_dev()) Skipped 5 previous similar messages [ 7386.268525] Lustre: server umount lustre-MDT0001 complete [ 7387.560762] LustreError: 344153:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0001: not available for connect from 192.168.202.34@tcp (no target). If you are running an HA pair check that the target is mounted on the other server. [ 7387.566142] LustreError: 344153:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 2 previous similar messages [ 7389.833267] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 7389.865732] Lustre: MGS: Regenerating lustre-MDT0001 log by user request: rc = 0 [ 7389.867932] Lustre: Found index 1 for lustre-MDT0001, updating log [ 7389.882118] Lustre: 341982:0:(mgs_llog.c:1348:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 7389.999501] Lustre: lustre-MDT0001: Imperative Recovery enabled, recovery window shrunk from 60-180 down to 60-180 [ 7391.391516] Lustre: DEBUG MARKER: oleg234-server.virtnet: executing set_default_debug -1 all [ 7395.301525] Lustre: lustre-MDT0001-lwp-OST0000: Connection restored to 0@lo (at 0@lo) [ 7399.909753] LustreError: 370892:0:(obd_mount.c:188:lustre_start_simple()) lustre-MDT0001-lwp-OST0000 attach error -17 [ 7399.912254] LustreError: 370892:0:(tgt_mount.c:669:lustre_lwp_setup()) lustre-MDT0001-lwp-OST0000: setup up failed: rc -17 [ 7399.914743] LustreError: 341986:0:(mgc_request.c:614:do_requeue()) failed processing log: -17 [ 7399.917122] LustreError: 341986:0:(mgc_request.c:614:do_requeue()) Skipped 1 previous similar message [ 7399.935899] LustreError: 370895:0:(lod_lov.c:212:lod_add_device()) lustre-MDT0001-osp-MDT0000: cannot connect to next dev lustre-MDT0001_UUID (-114) [ 7399.942970] LustreError: 370895:0:(obd_config.c:2001:class_config_llog_handler()) MGC192.168.202.134@tcp: cfg command failed: rc = -114 [ 7399.947648] Lustre: cmd=cf014 0:lustre-MDT0000-mdtlov 1:lustre-MDT0001_UUID 2:1 3:1 [ 7399.947648] [ 7707.620993] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 7707.623631] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 7707.624427] Lustre: Skipped 2 previous similar messages [ 7707.626043] Lustre: Skipped 3 previous similar messages [ 7707.627220] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 7710.851833] LustreError: 382397:0:(obd_class.h:479:obd_check_dev()) Device 9 not setup [ 7710.855313] LustreError: 382397:0:(obd_class.h:479:obd_check_dev()) Skipped 5 previous similar messages [ 7710.888522] Lustre: server umount lustre-MDT0000 complete [ 7712.099221] LustreError: 344864:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1770692467 with bad export cookie 16747541138929677079 [ 7712.103483] LustreError: 344864:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) Skipped 3 previous similar messages [ 7712.103957] LustreError: MGC192.168.202.134@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 7712.735926] LustreError: 341990:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 7712.740624] LustreError: 341990:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 2 previous similar messages [ 7717.856268] LustreError: 341990:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 7718.203699] LustreError: 382609:0:(obd_class.h:479:obd_check_dev()) Device 16 not setup [ 7718.205617] LustreError: 382609:0:(obd_class.h:479:obd_check_dev()) Skipped 5 previous similar messages [ 7718.256724] Lustre: server umount lustre-MDT0001 complete [ 7719.604028] Lustre: server umount lustre-OST0000 complete [ 7722.624722] Lustre: DEBUG MARKER: oleg234-server.virtnet: executing set_hostid [ 7725.638876] Lustre: DEBUG MARKER: oleg234-server.virtnet: executing load_modules_local [ 7737.903783] Lustre: DEBUG MARKER: == conf-sanity test 120: cross-target rename should not create bad symlinks ========================================================== 22:01:33 (1770692493) [ 7740.164561] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 7740.261454] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 7740.274297] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 7740.318268] Lustre: lustre-MDT0000: new disk, initializing [ 7740.345512] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 7740.354232] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 7741.699701] Lustre: DEBUG MARKER: oleg234-server.virtnet: executing set_default_debug -1 all [ 7745.832234] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 7745.873391] Lustre: 385328:0:(mgs_llog.c:1348:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 7745.898979] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [ 7745.900774] Lustre: Skipped 1 previous similar message [ 7745.950508] Lustre: lustre-MDT0001: new disk, initializing [ 7745.985340] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 7745.999243] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [ 7746.002601] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 7747.273581] Lustre: DEBUG MARKER: oleg234-server.virtnet: executing set_default_debug -1 all [ 7750.400230] Lustre: DEBUG MARKER: oleg234-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 7751.547360] Lustre: DEBUG MARKER: oleg234-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 7753.768974] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 7753.887107] Lustre: lustre-OST0000: new disk, initializing [ 7753.889375] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 7753.892947] Lustre: 387069:0:(osd_compat.c:1354:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 7753.926939] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 7755.641971] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [ 7755.645373] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 7755.678229] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 7755.860207] Lustre: DEBUG MARKER: oleg234-server.virtnet: executing set_default_debug -1 all [ 7758.920334] Lustre: DEBUG MARKER: oleg234-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 7765.988421] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 7765.992285] Lustre: Skipped 3 previous similar messages [ 7767.554871] LustreError: 387766:0:(obd_class.h:479:obd_check_dev()) Device 22 not setup [ 7767.558273] LustreError: 387766:0:(obd_class.h:479:obd_check_dev()) Skipped 11 previous similar messages [ 7767.596121] Lustre: server umount lustre-MDT0000 complete [ 7768.872334] LustreError: 385320:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1770692524 with bad export cookie 16747541138929679032 [ 7768.875434] LustreError: 385320:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) Skipped 3 previous similar messages [ 7768.875914] LustreError: MGC192.168.202.134@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 7768.876243] LustreError: 386120:0:(osp_object.c:617:osp_attr_get()) lustre-MDT0000-osp-MDT0001: osp_attr_get update error [0x200000401:0x2:0x0]: rc = -5 [ 7768.995730] Lustre: server umount lustre-MDT0001 complete [ 7779.494357] Lustre: server umount lustre-OST0000 complete [ 7788.758203] Lustre: DEBUG MARKER: oleg234-server.virtnet: executing set_hostid [ 7792.015131] Lustre: DEBUG MARKER: oleg234-server.virtnet: executing load_modules_local [ 7802.459712] Lustre: DEBUG MARKER: == conf-sanity test complete, duration 7510 sec ========== 22:02:37 (1770692557) [ 7803.012563] Lustre: DEBUG MARKER: === conf-sanity: start cleanup 22:02:38 (1770692558) === [ 7804.103984] Lustre: DEBUG MARKER: === conf-sanity: finish cleanup 22:02:39 (1770692559) === [ 7816.855897] Lustre: DEBUG MARKER: oleg234-server.virtnet: executing unload_modules_local [ 7817.854664] Key type lgssc unregistered [ 7817.978296] LNet: 393258:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 7817.980541] LNetError: 393258:0:(acceptor.c:264:lnet_acceptor_remove_socket()) Interface ens2 not found [ 7817.990343] LNet: Removed LNI 192.168.202.134@tcp [ 7818.302620] Key type .llcrypt unregistered [ 7818.303842] Key type ._llcrypt unregistered