[ 0.000000] Linux version 4.18.0rh8.10-debug (green@maintenance) (gcc version 8.5.0 20210514 (Red Hat 8.5.0-26) (GCC)) #2 SMP Mon Jul 14 01:24:22 EDT 2025 [ 0.000000] Command line: rd.shell root=nbd:192.168.200.253:rocky8.10:ext4:ro:-p,-b4096 ro crashkernel=256M panic=1 nomodeset ipmtu=9000 ip=dhcp rd.neednet=1 init_on_free=off mitigations=off console=ttyS1,115200 audit=0 [ 0.000000] x86/fpu: Supporting XSAVE feature 0x001: 'x87 floating point registers' [ 0.000000] x86/fpu: Supporting XSAVE feature 0x002: 'SSE registers' [ 0.000000] x86/fpu: Supporting XSAVE feature 0x004: 'AVX registers' [ 0.000000] x86/fpu: xstate_offset[2]: 576, xstate_sizes[2]: 256 [ 0.000000] x86/fpu: Enabled xstate features 0x7, context size is 832 bytes, using 'standard' format. [ 0.000000] signal: max sigframe size: 1776 [ 0.000000] BIOS-provided physical RAM map: [ 0.000000] BIOS-e820: [mem 0x0000000000000000-0x000000000009fbff] usable [ 0.000000] BIOS-e820: [mem 0x000000000009fc00-0x000000000009ffff] reserved [ 0.000000] BIOS-e820: [mem 0x00000000000f0000-0x00000000000fffff] reserved [ 0.000000] BIOS-e820: [mem 0x0000000000100000-0x00000000bffcdfff] usable [ 0.000000] BIOS-e820: [mem 0x00000000bffce000-0x00000000bfffffff] reserved [ 0.000000] BIOS-e820: [mem 0x00000000feffc000-0x00000000feffffff] reserved [ 0.000000] BIOS-e820: [mem 0x00000000fffc0000-0x00000000ffffffff] reserved [ 0.000000] BIOS-e820: [mem 0x0000000100000000-0x0000000146dfffff] usable [ 0.000000] NX (Execute Disable) protection: active [ 0.000000] SMBIOS 2.8 present. [ 0.000000] DMI: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.17.0-8.fc42 06/10/2025 [ 0.000000] Hypervisor detected: KVM [ 0.000000] kvm-clock: Using msrs 4b564d01 and 4b564d00 [ 0.000000] kvm-clock: using sched offset of 436572726 cycles [ 0.000000] clocksource: kvm-clock: mask: 0xffffffffffffffff max_cycles: 0x1cd42e4dffb, max_idle_ns: 881590591483 ns [ 0.000000] tsc: Detected 2399.998 MHz processor [ 0.000000] last_pfn = 0x146e00 max_arch_pfn = 0x400000000 [ 0.000000] x86/PAT: Configuration [0-7]: WB WC UC- UC WB WP UC- WT [ 0.000000] last_pfn = 0xbffce max_arch_pfn = 0x400000000 [ 0.000000] found SMP MP-table at [mem 0x000f54b0-0x000f54bf] [ 0.000000] RAMDISK: [mem 0xbcc54000-0xbffbffff] [ 0.000000] ACPI: Early table checksum verification disabled [ 0.000000] ACPI: RSDP 0x00000000000F52D0 000014 (v00 BOCHS ) [ 0.000000] ACPI: RSDT 0x00000000BFFE2439 000034 (v01 BOCHS BXPC 00000001 BXPC 00000001) [ 0.000000] ACPI: FACP 0x00000000BFFE22D5 000074 (v01 BOCHS BXPC 00000001 BXPC 00000001) [ 0.000000] ACPI: DSDT 0x00000000BFFE0040 002295 (v01 BOCHS BXPC 00000001 BXPC 00000001) [ 0.000000] ACPI: FACS 0x00000000BFFE0000 000040 [ 0.000000] ACPI: APIC 0x00000000BFFE2349 000090 (v03 BOCHS BXPC 00000001 BXPC 00000001) [ 0.000000] ACPI: HPET 0x00000000BFFE23D9 000038 (v01 BOCHS BXPC 00000001 BXPC 00000001) [ 0.000000] ACPI: WAET 0x00000000BFFE2411 000028 (v01 BOCHS BXPC 00000001 BXPC 00000001) [ 0.000000] ACPI: Reserving FACP table memory at [mem 0xbffe22d5-0xbffe2348] [ 0.000000] ACPI: Reserving DSDT table memory at [mem 0xbffe0040-0xbffe22d4] [ 0.000000] ACPI: Reserving FACS table memory at [mem 0xbffe0000-0xbffe003f] [ 0.000000] ACPI: Reserving APIC table memory at [mem 0xbffe2349-0xbffe23d8] [ 0.000000] ACPI: Reserving HPET table memory at [mem 0xbffe23d9-0xbffe2410] [ 0.000000] ACPI: Reserving WAET table memory at [mem 0xbffe2411-0xbffe2438] [ 0.000000] No NUMA configuration found [ 0.000000] Faking a node at [mem 0x0000000000000000-0x0000000146dfffff] [ 0.000000] NODE_DATA(0) allocated [mem 0x1465a3000-0x1465cdfff] [ 0.000000] Reserving 256MB of memory at 2752MB for crashkernel (System RAM: 4205MB) [ 0.000000] Zone ranges: [ 0.000000] DMA [mem 0x0000000000001000-0x0000000000ffffff] [ 0.000000] DMA32 [mem 0x0000000001000000-0x00000000ffffffff] [ 0.000000] Normal [mem 0x0000000100000000-0x0000000146dfffff] [ 0.000000] Device empty [ 0.000000] Movable zone start for each node [ 0.000000] Early memory node ranges [ 0.000000] node 0: [mem 0x0000000000001000-0x000000000009efff] [ 0.000000] node 0: [mem 0x0000000000100000-0x00000000bffcdfff] [ 0.000000] node 0: [mem 0x0000000100000000-0x0000000146dfffff] [ 0.000000] Zeroed struct page in unavailable ranges: 4756 pages [ 0.000000] Initmem setup node 0 [mem 0x0000000000001000-0x0000000146dfffff] [ 0.000000] ACPI: PM-Timer IO Port: 0x608 [ 0.000000] ACPI: LAPIC_NMI (acpi_id[0xff] dfl dfl lint[0x1]) [ 0.000000] IOAPIC[0]: apic_id 0, version 17, address 0xfec00000, GSI 0-23 [ 0.000000] ACPI: INT_SRC_OVR (bus 0 bus_irq 0 global_irq 2 dfl dfl) [ 0.000000] ACPI: INT_SRC_OVR (bus 0 bus_irq 5 global_irq 5 high level) [ 0.000000] ACPI: INT_SRC_OVR (bus 0 bus_irq 9 global_irq 9 high level) [ 0.000000] ACPI: INT_SRC_OVR (bus 0 bus_irq 10 global_irq 10 high level) [ 0.000000] ACPI: INT_SRC_OVR (bus 0 bus_irq 11 global_irq 11 high level) [ 0.000000] Using ACPI (MADT) for SMP configuration information [ 0.000000] ACPI: HPET id: 0x8086a201 base: 0xfed00000 [ 0.000000] TSC deadline timer available [ 0.000000] smpboot: Allowing 4 CPUs, 0 hotplug CPUs [ 0.000000] kvm-guest: KVM setup pv remote TLB flush [ 0.000000] kvm-guest: setup PV sched yield [ 0.000000] PM: Registered nosave memory: [mem 0x00000000-0x00000fff] [ 0.000000] PM: Registered nosave memory: [mem 0x0009f000-0x0009ffff] [ 0.000000] PM: Registered nosave memory: [mem 0x000a0000-0x000effff] [ 0.000000] PM: Registered nosave memory: [mem 0x000f0000-0x000fffff] [ 0.000000] PM: Registered nosave memory: [mem 0xbffce000-0xbfffffff] [ 0.000000] PM: Registered nosave memory: [mem 0xc0000000-0xfeffbfff] [ 0.000000] PM: Registered nosave memory: [mem 0xfeffc000-0xfeffffff] [ 0.000000] PM: Registered nosave memory: [mem 0xff000000-0xfffbffff] [ 0.000000] PM: Registered nosave memory: [mem 0xfffc0000-0xffffffff] [ 0.000000] [mem 0xc0000000-0xfeffbfff] available for PCI devices [ 0.000000] Booting paravirtualized kernel on KVM [ 0.000000] clocksource: refined-jiffies: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 1910969940391419 ns [ 0.000000] setup_percpu: NR_CPUS:8192 nr_cpumask_bits:4 nr_cpu_ids:4 nr_node_ids:1 [ 0.000000] percpu: Embedded 63 pages/cpu s221184 r8192 d28672 u524288 [ 0.000000] kvm-guest: PV spinlocks enabled [ 0.000000] PV qspinlock hash table entries: 256 (order: 0, 4096 bytes, linear) [ 0.000000] Built 1 zonelists, mobility grouping on. Total pages: 1059606 [ 0.000000] Policy zone: Normal [ 0.000000] Kernel command line: rd.shell root=nbd:192.168.200.253:rocky8.10:ext4:ro:-p,-b4096 ro crashkernel=256M panic=1 nomodeset ipmtu=9000 ip=dhcp rd.neednet=1 init_on_free=off mitigations=off console=ttyS1,115200 audit=0 [ 0.000000] Specific versions of hardware are certified with Red Hat Enterprise Linux 8. Please see the list of hardware certified with Red Hat Enterprise Linux 8 at https://catalog.redhat.com. [ 0.000000] audit: disabled (until reboot) [ 0.000000] software IO TLB: area num 4. [ 0.000000] Memory: 2829652K/4306352K available (18435K kernel code, 11221K rwdata, 7248K rodata, 2908K init, 18040K bss, 524584K reserved, 0K cma-reserved) [ 0.000000] SLUB: HWalign=64, Order=0-3, MinObjects=0, CPUs=4, Nodes=1 [ 0.000000] kmemleak: Kernel memory leak detector disabled [ 0.000000] ftrace: allocating 41240 entries in 162 pages [ 0.000000] ftrace: allocated 162 pages with 3 groups [ 0.000000] rcu: Hierarchical RCU implementation. [ 0.000000] rcu: RCU event tracing is enabled. [ 0.000000] rcu: RCU restricting CPUs from NR_CPUS=8192 to nr_cpu_ids=4. [ 0.000000] rcu: RCU callback double-/use-after-free debug enabled. [ 0.000000] Rude variant of Tasks RCU enabled. [ 0.000000] Tracing variant of Tasks RCU enabled. [ 0.000000] rcu: RCU calculated value of scheduler-enlistment delay is 100 jiffies. [ 0.000000] rcu: Adjusting geometry for rcu_fanout_leaf=16, nr_cpu_ids=4 [ 0.000000] NR_IRQS: 524544, nr_irqs: 456, preallocated irqs: 16 [ 0.000000] random: get_random_bytes called from start_kernel+0x622/0x9a8 with crng_init=0 [ 0.001000] Console: colour *CGA 80x25 [ 0.001000] printk: console [ttyS1] enabled [ 0.001000] ACPI: Core revision 20220331 [ 0.001000] clocksource: hpet: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 19112604467 ns [ 0.001009] APIC: Switch to symmetric I/O mode setup [ 0.002276] x2apic enabled [ 0.003004] Switched APIC routing to physical x2apic. [ 0.004008] kvm-guest: setup PV IPIs [ 0.006927] ..TIMER: vector=0x30 apic1=0 pin1=2 apic2=-1 pin2=-1 [ 0.007000] clocksource: tsc-early: mask: 0xffffffffffffffff max_cycles: 0x229835b7123, max_idle_ns: 440795242976 ns [ 0.007010] Calibrating delay loop (skipped) preset value.. 4799.99 BogoMIPS (lpj=2399998) [ 0.008005] pid_max: default: 32768 minimum: 301 [ 0.009086] LSM: Security Framework initializing [ 0.010030] Yama: becoming mindful. [ 0.011022] SELinux: Initializing. [ 0.013047] *** VALIDATE selinux *** [ 0.019608] Dentry cache hash table entries: 1048576 (order: 11, 8388608 bytes, vmalloc) [ 0.022599] Inode-cache hash table entries: 524288 (order: 10, 4194304 bytes, vmalloc) [ 0.023086] Mount-cache hash table entries: 16384 (order: 5, 131072 bytes, vmalloc) [ 0.024068] Mountpoint-cache hash table entries: 16384 (order: 5, 131072 bytes, vmalloc) [ 0.025063] *** VALIDATE tmpfs *** [ 0.026322] *** VALIDATE proc *** [ 0.027150] *** VALIDATE cgroup *** [ 0.028006] *** VALIDATE cgroup2 *** [ 0.030112] x86/cpu: User Mode Instruction Prevention (UMIP) activated [ 0.031101] Last level iTLB entries: 4KB 0, 2MB 0, 4MB 0 [ 0.032003] Last level dTLB entries: 4KB 0, 2MB 0, 4MB 0, 1GB 0 [ 0.033020] Spectre V2 : User space: Vulnerable [ 0.034004] Speculative Store Bypass: Vulnerable [ 0.036921] debug: unmapping init [mem 0xffffffff84c59000-0xffffffff84c60fff] [ 0.038116] smpboot: CPU0: Intel(R) Xeon(R) CPU E5-2695 v2 @ 2.40GHz (family: 0x6, model: 0x3e, stepping: 0x4) [ 0.039410] Performance Events: IvyBridge events, full-width counters, Intel PMU driver. [ 0.040010] ... version: 2 [ 0.041007] ... bit width: 48 [ 0.042007] ... generic registers: 4 [ 0.043008] ... value mask: 0000ffffffffffff [ 0.044007] ... max period: 00007fffffffffff [ 0.045007] ... fixed-purpose events: 3 [ 0.046006] ... event mask: 000000070000000f [ 0.047192] rcu: Hierarchical SRCU implementation. [ 0.049095] smp: Bringing up secondary CPUs ... [ 0.050366] x86: Booting SMP configuration: [ 0.051018] .... node #0, CPUs: #1 #2 #3 [ 0.053517] smp: Brought up 1 node, 4 CPUs [ 0.055008] smpboot: Max logical packages: 1 [ 0.056011] smpboot: Total of 4 processors activated (19199.98 BogoMIPS) [ 0.244870] node 0 deferred pages initialised in 187ms [ 0.247119] devtmpfs: initialized [ 0.248158] x86/mm: Memory block size: 128MB [ 0.250220] gcov: version magic: 0x41383552 [ 0.251552] clocksource: jiffies: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 1911260446275000 ns [ 0.252067] futex hash table entries: 1024 (order: 4, 65536 bytes, vmalloc) [ 0.253174] pinctrl core: initialized pinctrl subsystem [ 0.254106] [ 0.254559] ************************************************************* [ 0.255007] ** NOTICE NOTICE NOTICE NOTICE NOTICE NOTICE NOTICE ** [ 0.256009] ** ** [ 0.257008] ** IOMMU DebugFS SUPPORT HAS BEEN ENABLED IN THIS KERNEL ** [ 0.258008] ** ** [ 0.259010] ** This means that this kernel is built to expose internal ** [ 0.260008] ** IOMMU data structures, which may compromise security on ** [ 0.261009] ** your system. ** [ 0.262012] ** ** [ 0.263010] ** If you see this message and you are not debugging the ** [ 0.264010] ** kernel, report this immediately to your vendor! ** [ 0.265009] ** ** [ 0.266006] ** NOTICE NOTICE NOTICE NOTICE NOTICE NOTICE NOTICE ** [ 0.267009] ************************************************************* [ 0.268482] NET: Registered protocol family 16 [ 0.269319] DMA: preallocated 512 KiB GFP_KERNEL pool for atomic allocations [ 0.270031] DMA: preallocated 512 KiB GFP_KERNEL|GFP_DMA pool for atomic allocations [ 0.271031] DMA: preallocated 512 KiB GFP_KERNEL|GFP_DMA32 pool for atomic allocations [ 0.272309] cpuidle: using governor menu [ 0.273000] acpiphp: ACPI Hot Plug PCI Controller Driver version: 0.5 [ 0.275455] PCI: Using configuration type 1 for base access [ 0.276090] core: PMU erratum BJ122, BV98, HSD29 worked around, HT is on [ 0.283112] HugeTLB registered 1.00 GiB page size, pre-allocated 0 pages [ 0.284018] HugeTLB registered 2.00 MiB page size, pre-allocated 0 pages [ 0.285132] cryptd: max_cpu_qlen set to 1000 [ 0.286213] ACPI: Added _OSI(Module Device) [ 0.287000] ACPI: Added _OSI(Processor Device) [ 0.289015] ACPI: Added _OSI(3.0 _SCP Extensions) [ 0.289906] ACPI: Added _OSI(Processor Aggregator Device) [ 0.293360] ACPI: 1 ACPI AML tables successfully acquired and loaded [ 0.299369] ACPI: Interpreter enabled [ 0.300040] ACPI: PM: (supports S0 S3 S4 S5) [ 0.300978] ACPI: Using IOAPIC for interrupt routing [ 0.302054] PCI: Using host bridge windows from ACPI; if necessary, use "pci=nocrs" and report a bug [ 0.304231] ACPI: Enabled 2 GPEs in block 00 to 0F [ 0.312287] ACPI: PCI Root Bridge [PCI0] (domain 0000 [bus 00-ff]) [ 0.314020] acpi PNP0A03:00: _OSC: OS supports [ASPM ClockPM Segments MSI HPX-Type3] [ 0.316011] acpi PNP0A03:00: _OSC: not requesting OS control; OS requires [ExtendedConfig ASPM ClockPM MSI] [ 0.319066] acpi PNP0A03:00: fail to add MMCONFIG information, can't access extended PCI configuration space under this bridge. [ 0.322032] acpiphp: Slot [2] registered [ 0.322814] acpiphp: Slot [5] registered [ 0.323064] acpiphp: Slot [6] registered [ 0.323936] acpiphp: Slot [7] registered [ 0.325050] acpiphp: Slot [8] registered [ 0.325811] acpiphp: Slot [9] registered [ 0.327076] acpiphp: Slot [10] registered [ 0.328050] acpiphp: Slot [3] registered [ 0.328991] acpiphp: Slot [4] registered [ 0.331081] acpiphp: Slot [11] registered [ 0.332063] acpiphp: Slot [12] registered [ 0.333088] acpiphp: Slot [13] registered [ 0.335068] acpiphp: Slot [14] registered [ 0.336071] acpiphp: Slot [15] registered [ 0.338069] acpiphp: Slot [16] registered [ 0.339058] acpiphp: Slot [17] registered [ 0.340084] acpiphp: Slot [18] registered [ 0.342066] acpiphp: Slot [19] registered [ 0.342884] acpiphp: Slot [20] registered [ 0.344048] acpiphp: Slot [21] registered [ 0.345021] acpiphp: Slot [22] registered [ 0.347090] acpiphp: Slot [23] registered [ 0.348050] acpiphp: Slot [24] registered [ 0.348860] acpiphp: Slot [25] registered [ 0.350063] acpiphp: Slot [26] registered [ 0.351071] acpiphp: Slot [27] registered [ 0.353049] acpiphp: Slot [28] registered [ 0.353811] acpiphp: Slot [29] registered [ 0.355057] acpiphp: Slot [30] registered [ 0.356049] acpiphp: Slot [31] registered [ 0.356905] PCI host bridge to bus 0000:00 [ 0.357015] pci_bus 0000:00: root bus resource [io 0x0000-0x0cf7 window] [ 0.359009] pci_bus 0000:00: root bus resource [io 0x0d00-0xffff window] [ 0.360023] pci_bus 0000:00: root bus resource [mem 0x000a0000-0x000bffff window] [ 0.362017] pci_bus 0000:00: root bus resource [mem 0xc0000000-0xfebfffff window] [ 0.363009] pci_bus 0000:00: root bus resource [mem 0xe0000000000-0xe007fffffff window] [ 0.365031] pci_bus 0000:00: root bus resource [bus 00-ff] [ 0.366213] pci 0000:00:00.0: [8086:1237] type 00 class 0x060000 [ 0.368858] pci 0000:00:01.0: [8086:7000] type 00 class 0x060100 [ 0.370916] pci 0000:00:01.1: [8086:7010] type 00 class 0x010180 [ 0.377778] pci 0000:00:01.1: reg 0x20: [io 0xc320-0xc32f] [ 0.381079] pci 0000:00:01.1: legacy IDE quirk: reg 0x10: [io 0x01f0-0x01f7] [ 0.383008] pci 0000:00:01.1: legacy IDE quirk: reg 0x14: [io 0x03f6] [ 0.384010] pci 0000:00:01.1: legacy IDE quirk: reg 0x18: [io 0x0170-0x0177] [ 0.386011] pci 0000:00:01.1: legacy IDE quirk: reg 0x1c: [io 0x0376] [ 0.387442] pci 0000:00:01.3: [8086:7113] type 00 class 0x068000 [ 0.389596] pci 0000:00:01.3: quirk: [io 0x0600-0x063f] claimed by PIIX4 ACPI [ 0.392038] pci 0000:00:01.3: quirk: [io 0x0700-0x070f] claimed by PIIX4 SMB [ 0.395530] pci 0000:00:02.0: [1af4:1000] type 00 class 0x020000 [ 0.398009] pci 0000:00:02.0: reg 0x10: [io 0xc300-0xc31f] [ 0.406009] pci 0000:00:02.0: reg 0x20: [mem 0xe0000000000-0xe0000003fff 64bit pref] [ 0.410007] pci 0000:00:02.0: reg 0x30: [mem 0xfeb80000-0xfebbffff pref] [ 0.413338] pci 0000:00:05.0: [1af4:1001] type 00 class 0x010000 [ 0.417008] pci 0000:00:05.0: reg 0x10: [io 0xc000-0xc07f] [ 0.421008] pci 0000:00:05.0: reg 0x14: [mem 0xfebc0000-0xfebc0fff] [ 0.431010] pci 0000:00:05.0: reg 0x20: [mem 0xe0000004000-0xe0000007fff 64bit pref] [ 0.436904] pci 0000:00:06.0: [1af4:1001] type 00 class 0x010000 [ 0.442009] pci 0000:00:06.0: reg 0x10: [io 0xc080-0xc0ff] [ 0.447008] pci 0000:00:06.0: reg 0x14: [mem 0xfebc1000-0xfebc1fff] [ 0.457011] pci 0000:00:06.0: reg 0x20: [mem 0xe0000008000-0xe000000bfff 64bit pref] [ 0.466373] pci 0000:00:07.0: [1af4:1001] type 00 class 0x010000 [ 0.471008] pci 0000:00:07.0: reg 0x10: [io 0xc100-0xc17f] [ 0.475010] pci 0000:00:07.0: reg 0x14: [mem 0xfebc2000-0xfebc2fff] [ 0.484010] pci 0000:00:07.0: reg 0x20: [mem 0xe000000c000-0xe000000ffff 64bit pref] [ 0.489984] pci 0000:00:08.0: [1af4:1001] type 00 class 0x010000 [ 0.493008] pci 0000:00:08.0: reg 0x10: [io 0xc180-0xc1ff] [ 0.496007] pci 0000:00:08.0: reg 0x14: [mem 0xfebc3000-0xfebc3fff] [ 0.504008] pci 0000:00:08.0: reg 0x20: [mem 0xe0000010000-0xe0000013fff 64bit pref] [ 0.510055] pci 0000:00:09.0: [1af4:1001] type 00 class 0x010000 [ 0.514010] pci 0000:00:09.0: reg 0x10: [io 0xc200-0xc27f] [ 0.517008] pci 0000:00:09.0: reg 0x14: [mem 0xfebc4000-0xfebc4fff] [ 0.525020] pci 0000:00:09.0: reg 0x20: [mem 0xe0000014000-0xe0000017fff 64bit pref] [ 0.532816] pci 0000:00:0a.0: [1af4:1001] type 00 class 0x010000 [ 0.536009] pci 0000:00:0a.0: reg 0x10: [io 0xc280-0xc2ff] [ 0.540008] pci 0000:00:0a.0: reg 0x14: [mem 0xfebc5000-0xfebc5fff] [ 0.549010] pci 0000:00:0a.0: reg 0x20: [mem 0xe0000018000-0xe000001bfff 64bit pref] [ 0.557659] ACPI: PCI: Interrupt link LNKA configured for IRQ 10 [ 0.559212] ACPI: PCI: Interrupt link LNKB configured for IRQ 10 [ 0.561995] ACPI: PCI: Interrupt link LNKC configured for IRQ 11 [ 0.563188] ACPI: PCI: Interrupt link LNKD configured for IRQ 11 [ 0.564104] ACPI: PCI: Interrupt link LNKS configured for IRQ 9 [ 0.568029] iommu: Default domain type: Passthrough [ 0.569383] SCSI subsystem initialized [ 0.570069] ACPI: bus type USB registered [ 0.570946] usbcore: registered new interface driver usbfs [ 0.572032] usbcore: registered new interface driver hub [ 0.574039] usbcore: registered new device driver usb [ 0.576101] pps_core: LinuxPPS API ver. 1 registered [ 0.577006] pps_core: Software ver. 5.3.6 - Copyright 2005-2007 Rodolfo Giometti [ 0.580054] PTP clock support registered [ 0.581117] EDAC MC: Ver: 3.0.0 [ 0.582297] PCI: Using ACPI for IRQ routing [ 0.583656] NetLabel: Initializing [ 0.585007] NetLabel: domain hash size = 128 [ 0.585938] NetLabel: protocols = UNLABELED CIPSOv4 CALIPSO [ 0.587046] NetLabel: unlabeled traffic allowed by default [ 0.588097] vgaarb: loaded [ 0.589301] hpet0: at MMIO 0xfed00000, IRQs 2, 8, 0 [ 0.591010] hpet0: 3 comparators, 64-bit 100.000000 MHz counter [ 0.596380] clocksource: Switched to clocksource kvm-clock [ 0.683991] VFS: Disk quotas dquot_6.6.0 [ 0.684910] VFS: Dquot-cache hash table entries: 512 (order 0, 4096 bytes) [ 0.686301] *** VALIDATE ramfs *** [ 0.686964] *** VALIDATE hugetlbfs *** [ 0.687862] pnp: PnP ACPI init [ 0.690015] pnp: PnP ACPI: found 6 devices [ 0.702819] clocksource: acpi_pm: mask: 0xffffff max_cycles: 0xffffff, max_idle_ns: 2085701024 ns [ 0.704633] pci_bus 0000:00: resource 4 [io 0x0000-0x0cf7 window] [ 0.705785] pci_bus 0000:00: resource 5 [io 0x0d00-0xffff window] [ 0.706928] pci_bus 0000:00: resource 6 [mem 0x000a0000-0x000bffff window] [ 0.708269] pci_bus 0000:00: resource 7 [mem 0xc0000000-0xfebfffff window] [ 0.709611] pci_bus 0000:00: resource 8 [mem 0xe0000000000-0xe007fffffff window] [ 0.711104] NET: Registered protocol family 2 [ 0.712527] IP idents hash table entries: 131072 (order: 8, 1048576 bytes, vmalloc) [ 0.715571] tcp_listen_portaddr_hash hash table entries: 4096 (order: 5, 163840 bytes, vmalloc) [ 0.717504] TCP established hash table entries: 65536 (order: 7, 524288 bytes, vmalloc) [ 0.720936] TCP bind hash table entries: 65536 (order: 9, 2097152 bytes, vmalloc) [ 0.722740] TCP: Hash tables configured (established 65536 bind 65536) [ 0.724227] MPTCP token hash table entries: 8192 (order: 6, 393216 bytes, vmalloc) [ 0.725897] UDP hash table entries: 4096 (order: 6, 393216 bytes, vmalloc) [ 0.727359] UDP-Lite hash table entries: 4096 (order: 6, 393216 bytes, vmalloc) [ 0.729031] NET: Registered protocol family 1 [ 0.730523] RPC: Registered named UNIX socket transport module. [ 0.731770] RPC: Registered udp transport module. [ 0.732746] RPC: Registered tcp transport module. [ 0.733712] RPC: Registered tcp NFSv4.1 backchannel transport module. [ 0.735087] NET: Registered protocol family 44 [ 0.736021] pci 0000:00:00.0: Limiting direct PCI/PCI transfers [ 0.737287] pci 0000:00:01.0: PIIX3: Enabling Passive Release [ 0.738478] pci 0000:00:01.0: Activating ISA DMA hang workarounds [ 0.739817] PCI: CLS 0 bytes, default 64 [ 0.740756] Unpacking initramfs... [ 1.962990] debug: unmapping init [mem 0xffff94a93cc54000-0xffff94a93ffbffff] [ 1.965663] PCI-DMA: Using software bounce buffering for IO (SWIOTLB) [ 1.967071] software IO TLB: mapped [mem 0x00000000a8000000-0x00000000ac000000] (64MB) [ 1.968900] clocksource: tsc: mask: 0xffffffffffffffff max_cycles: 0x229835b7123, max_idle_ns: 440795242976 ns [ 2.409944] Initialise system trusted keyrings [ 2.412600] Key type blacklist registered [ 2.414238] workingset: timestamp_bits=36 max_order=20 bucket_order=0 [ 2.420449] zbud: loaded [ 2.422212] *** VALIDATE nfs *** [ 2.422899] *** VALIDATE nfs4 *** [ 2.423818] pstore: using deflate compression [ 2.425448] Platform Keyring initialized [ 2.490550] NET: Registered protocol family 38 [ 2.491639] Key type asymmetric registered [ 2.492510] Asymmetric key parser 'x509' registered [ 2.493494] Block layer SCSI generic (bsg) driver version 0.4 loaded (major 247) [ 2.495035] io scheduler mq-deadline registered [ 2.496083] io scheduler kyber registered [ 2.497067] io scheduler bfq registered [ 2.498634] atomic64_test: passed for x86-64 platform with CX8 and with SSE [ 2.501287] shpchp: Standard Hot Plug PCI Controller Driver version: 0.4 [ 2.502996] input: Power Button as /devices/LNXSYSTM:00/LNXPWRBN:00/input/input0 [ 2.504582] ACPI: Power Button [PWRF] [ 2.508035] ACPI: \_SB_.LNKB: Enabled at IRQ 10 [ 2.512609] ACPI: \_SB_.LNKA: Enabled at IRQ 11 [ 2.519676] ACPI: \_SB_.LNKC: Enabled at IRQ 11 [ 2.523533] ACPI: \_SB_.LNKD: Enabled at IRQ 10 [ 2.531320] Serial: 8250/16550 driver, 4 ports, IRQ sharing enabled [ 2.556272] 00:03: ttyS1 at I/O 0x2f8 (irq = 3, base_baud = 115200) is a 16550A [ 2.581974] 00:04: ttyS0 at I/O 0x3f8 (irq = 4, base_baud = 115200) is a 16550A [ 2.585018] Non-volatile memory driver v1.3 [ 2.586725] Linux agpgart interface v0.103 [ 2.610578] virtio_blk virtio1: [vda] 134040 512-byte logical blocks (68.6 MB/65.4 MiB) [ 2.612921] vda: detected capacity change from 0 to 68628480 [ 2.624553] virtio_blk virtio2: [vdb] 2097152 512-byte logical blocks (1.07 GB/1.00 GiB) [ 2.626913] vdb: detected capacity change from 0 to 1073741824 [ 2.638449] virtio_blk virtio3: [vdc] 5120000 512-byte logical blocks (2.62 GB/2.44 GiB) [ 2.640718] vdc: detected capacity change from 0 to 2621440000 [ 2.651799] virtio_blk virtio4: [vdd] 5120000 512-byte logical blocks (2.62 GB/2.44 GiB) [ 2.653786] vdd: detected capacity change from 0 to 2621440000 [ 2.663848] virtio_blk virtio5: [vde] 8388608 512-byte logical blocks (4.29 GB/4.00 GiB) [ 2.665841] vde: detected capacity change from 0 to 4294967296 [ 2.676089] virtio_blk virtio6: [vdf] 8388608 512-byte logical blocks (4.29 GB/4.00 GiB) [ 2.677992] vdf: detected capacity change from 0 to 4294967296 [ 2.682667] libphy: Fixed MDIO Bus: probed [ 2.686692] usbcore: registered new interface driver usbserial_generic [ 2.688710] usbserial: USB Serial support registered for generic [ 2.690748] i8042: PNP: PS/2 Controller [PNP0303:KBD,PNP0f13:MOU] at 0x60,0x64 irq 1,12 [ 2.693969] serio: i8042 KBD port at 0x60,0x64 irq 1 [ 2.695558] serio: i8042 AUX port at 0x60,0x64 irq 12 [ 2.697670] mousedev: PS/2 mouse device common for all mice [ 2.699279] rtc_cmos 00:05: RTC can wake from S4 [ 2.699473] input: AT Translated Set 2 keyboard as /devices/platform/i8042/serio0/input/input1 [ 2.702160] rtc_cmos 00:05: registered as rtc0 [ 2.704996] input: VirtualPS/2 VMware VMMouse as /devices/platform/i8042/serio1/input/input4 [ 2.707342] rtc_cmos 00:05: alarms up to one day, y3k, 242 bytes nvram, hpet irqs [ 2.708377] input: VirtualPS/2 VMware VMMouse as /devices/platform/i8042/serio1/input/input3 [ 2.709298] intel_pstate: CPU model not supported [ 2.712977] hid: raw HID events driver (C) Jiri Kosina [ 2.714357] usbcore: registered new interface driver usbhid [ 2.715777] usbhid: USB HID core driver [ 2.716831] drop_monitor: Initializing network drop monitor service [ 2.718660] Initializing XFRM netlink socket [ 2.719985] NET: Registered protocol family 10 [ 2.721915] Segment Routing with IPv6 [ 2.722895] NET: Registered protocol family 17 [ 2.724214] mpls_gso: MPLS GSO support [ 2.728502] RAS: Correctable Errors collector initialized. [ 2.729624] AVX version of gcm_enc/dec engaged. [ 2.730564] AES CTR mode by8 optimization enabled [ 2.772812] sched_clock: Marking stable (2772794742, 0)->(3538134657, -765339915) [ 2.776465] registered taskstats version 1 [ 2.778903] Loading compiled-in X.509 certificates [ 2.781267] zswap: loaded using pool lzo/zbud [ 2.799909] Key type big_key registered [ 2.807719] Key type encrypted registered [ 2.808816] ima: No TPM chip found, activating TPM-bypass! [ 2.809962] ima: Allocated hash algorithm: sha1 [ 2.810886] ima: No architecture policies found [ 2.812439] evm: Initialising EVM extended attributes: [ 2.813720] evm: security.selinux [ 2.814331] evm: security.ima [ 2.814972] evm: security.capability [ 2.815817] evm: HMAC attrs: 0x1 [ 2.817475] rtc_cmos 00:05: setting system clock to 2026-01-02 20:16:23 UTC (1767384983) [ 2.822353] debug: unmapping init [mem 0xffffffff85c03000-0xffffffff85dfffff] [ 2.824049] debug: unmapping init [mem 0xffffffff84982000-0xffffffff84c58fff] [ 2.833082] Write protecting the kernel read-only data: 28672k [ 2.836016] debug: unmapping init [mem 0xffffffff83003000-0xffffffff831fffff] [ 2.837848] debug: unmapping init [mem 0xffffffff83914000-0xffffffff839fffff] [ 2.867069] systemd[1]: systemd 239 (239-82.el8_10.5) running in system mode. (+PAM +AUDIT +SELINUX +IMA -APPARMOR +SMACK +SYSVINIT +UTMP +LIBCRYPTSETUP +GCRYPT +GNUTLS +ACL +XZ +LZ4 +SECCOMP +BLKID +ELFUTILS +KMOD +IDN2 -IDN +PCRE2 default-hierarchy=legacy) [ 2.874364] systemd[1]: Detected virtualization kvm. [ 2.875516] systemd[1]: Detected architecture x86-64. [ 2.876683] systemd[1]: Running in initial RAM disk. Welcome to Rocky Linux 8.10 (Green Obsidian) dracut-049-233.git20240115.el8 (Initramfs)! [ 2.901360] systemd[1]: No hostname configured. [ 2.902933] systemd[1]: Set hostname to . [ 2.904933] random: systemd: uninitialized urandom read (16 bytes read) [ 2.906255] systemd[1]: Initializing machine ID from random generator. [ 2.941321] random: ln: uninitialized urandom read (6 bytes read) [ 3.017123] random: systemd: uninitialized urandom read (16 bytes read) [ 3.018530] systemd[1]: Reached target Swap. [ OK ] Reached target Swap. [ 3.020722] systemd[1]: Reached target Slices. [ OK ] Reached target Slices. [ 3.022876] systemd[1]: Reached target Initrd Root Device. [ OK ] Reached target Initrd Root Device. [ OK ] Listening on udev Kernel Socket. [ OK ] Started Dispatch Password Requests to Console Directory Watch. [ OK ] Reached target Local Encrypted Volumes. [ OK ] Reached target Paths. [ OK ] Reached target Local File Systems. [ OK ] Listening on Journal Socket (/dev/log). [ OK ] Reached target Timers. [ OK ] Listening on Journal Socket. Starting Setup Virtual Console... Starting Apply Kernel Variables... Starting Journal Service... [ OK ] Started Memstrack Anylazing Service. Starting Create list of required st…ce nodes for the current kernel... [ OK ] Listening on udev Control Socket. [ OK ] Reached target Sockets. Starting Create Volatile Files and Directories... [ OK ] Started Setup Virtual Console. [ OK ] Started Apply Kernel Variables. [ OK ] Started Create list of required sta…vice nodes for the current kernel. [ OK ] Started Create Volatile Files and Directories. Starting Create Static Device Nodes in /dev... Starting dracut cmdline hook... [ OK ] Started Journal Service. [ OK ] Started Create Static Device Nodes in /dev. [ OK ] Started dracut cmdline hook. Starting dracut pre-udev hook... [ 3.491378] device-mapper: uevent: version 1.0.3 [ 3.493106] device-mapper: ioctl: 4.46.0-ioctl (2022-02-22) initialised: dm-devel@redhat.com [ OK ] Started dracut pre-udev hook. Starting udev Kernel Device Manager... [ OK ] Started udev Kernel Device Manager. Starting dracut pre-trigger hook... [ OK ] Started dracut pre-trigger hook. Starting udev Coldplug all Devices... Mounting Kernel Configuration File System... [ OK ] Mounted Kernel Configuration File System. [ OK ] Started udev Coldplug all Devices. Starting dracut initqueue hook... [ OK ] Reached target System Initialization. [ OK ] Reached target Basic System. [ OK ] Started Hardware RNG Entropy Gatherer Daemon. [ 4.100509] virtio_net virtio0 ens2: renamed from eth0 [ 4.115505] random: fast init done [ 4.172447] scsi host0: ata_piix [ 4.188772] scsi host1: ata_piix [ 4.190072] ata1: PATA max MWDMA2 cmd 0x1f0 ctl 0x3f6 bmdma 0xc320 irq 14 [ 4.191883] ata2: PATA max MWDMA2 cmd 0x170 ctl 0x376 bmdma 0xc328 irq 15 [ 8.449194] dracut-initqueue[591]: RTNETLINK answers: File exists [ 9.321316] random: crng init done [ 9.322514] random: 7 urandom warning(s) missed due to ratelimiting Starting nbd nbd0... [ OK ] Started nbd nbd0. [ OK ] Started dracut initqueue hook. [ OK ] Reached target Remote File Systems (Pre). [ OK ] Reached target Remote File Systems. Mounting /sysroot... [ 9.440477] EXT4-fs (nbd0): mounted filesystem with ordered data mode. Opts: (null) [ OK ] Mounted /sysroot. [ OK ] Reached target Initrd Root File System. Starting Reload Configuration from the Real Root... [ OK ] Started Reload Configuration from the Real Root. [ OK ] Reached target Initrd File Systems. [ OK ] Reached target Initrd Default Target. Starting dracut pre-pivot and cleanup hook... [ OK ] Started dracut pre-pivot and cleanup hook. Starting Cleaning Up and Shutting Down Daemons... [ OK ] Stopped target Timers. [ OK ] Stopped dracut pre-pivot and cleanup hook. [ OK ] Stopped target Initrd Default Target. [ OK ] Stopped target Remote File Systems. [ OK ] Stopped target Remote File Systems (Pre). [ OK ] Stopped dracut initqueue hook. Stopping Hardware RNG Entropy Gatherer Daemon... [ OK ] Stopped target Initrd Root Device. [ OK ] Stopped Hardware RNG Entropy Gatherer Daemon. [ OK ] Stopped target Basic System. [ OK ] Stopped target Sockets. [ OK ] Stopped target Paths. [ OK ] Stopped target Slices. [ OK ] Stopped target System Initialization. [ OK ] Stopped udev Coldplug all Devices. [ OK ] Stopped dracut pre-trigger hook. Stopping udev Kernel Device Manager... [ OK ] Stopped target Local Encrypted Volumes. [ OK ] Stopped Dispatch Password Requests to Console Directory Watch. [ OK ] Stopped target Swap. [ OK ] Stopped Create Volatile Files and Directories. [ OK ] Stopped target Local File Systems. [ OK ] Stopped Apply Kernel Variables. [ OK ] Started Cleaning Up and Shutting Down Daemons. [ OK ] Stopped udev Kernel Device Manager. [ OK ] Stopped dracut pre-udev hook. [ OK ] Stopped dracut cmdline hook. [ OK ] Stopped Create Static Device Nodes in /dev. [ OK ] Stopped Create list of required sta…vice nodes for the current kernel. [ OK ] Closed udev Kernel Socket. [ OK ] Closed udev Control Socket. Starting Cleanup udevd DB... [ OK ] Started Cleanup udevd DB. [ OK ] Reached target Switch Root. Starting Switch Root... [ 10.300926] printk: systemd: 25 output lines suppressed due to ratelimiting [ 10.463112] SELinux: Disabled at runtime. [ 10.511335] systemd[1]: systemd 239 (239-82.el8_10.5) running in system mode. (+PAM +AUDIT +SELINUX +IMA -APPARMOR +SMACK +SYSVINIT +UTMP +LIBCRYPTSETUP +GCRYPT +GNUTLS +ACL +XZ +LZ4 +SECCOMP +BLKID +ELFUTILS +KMOD +IDN2 -IDN +PCRE2 default-hierarchy=legacy) [ 10.517259] systemd[1]: Detected virtualization kvm. [ 10.518449] systemd[1]: Detected architecture x86-64. Welcome to Rocky Linux 8.10 (Green Obsidian)! [ 10.837641] systemd[1]: initrd-switch-root.service: Succeeded. [ 10.841313] systemd[1]: Stopped Switch Root. [ OK ] Stopped Switch Root. [ 10.845737] systemd[1]: systemd-journald.service: Service has no hold-off time (RestartSec=0), scheduling restart. [ 10.848811] systemd[1]: systemd-journald.service: Scheduled restart job, restart counter is at 1. [ 10.851953] systemd[1]: Stopped Journal Service. [ OK ] Stopped Journal Service. [ 10.858732] systemd[1]: Starting Journal Service... Starting Journal Service... [ 10.862869] systemd[1]: Created slice system-serial\x2dgetty.slice. [ OK ] Created slice system-serial\x2dgetty.slice. [ OK ] Listening on RPCbind Server Activation Socket. [ OK ] Reached target RPC Port Mapper. [ OK ] Started Dispatch Password Requests to Console Directory Watch. Mounting Kernel Debug File System... [ OK ] Listening on Process Core Dump Socket. [ OK ] Listening on initctl Compatibility Named Pipe. [ OK ] Created slice system-getty.slice. [ OK ] Listening on udev Control Socket. [ OK ] Created slice system-sshd\x2dkeygen.slice. [ OK ] Stopped target Switch Root. [ OK ] Stopped target Initrd File Systems. [ OK ] Stopped target Initrd Root File System. Mounting POSIX Message Queue File System... Starting Remount Root and Kernel File Systems... [ OK ] Listening on udev Kernel Socket. Starting udev Coldplug all Devices... [ OK ] Reached target rpc_pipefs.target. Starting Create list of required st…ce nodes for the current kernel... [FAILED] Failed to set up automount Arbitrar…rmats File System Automount Point. See 'systemctl status proc-sys-fs-binfmt_misc.automount' for details. Activating swap /dev/disk/by-label/SWAP... Starting Apply Kernel Variables... [ OK ] Started Forward Password Requests to Wall Directory Watch. [ OK ] Reached target Local Encrypted Volumes. [ OK ] Reached target Paths. Mounting Huge Pages File System... [ 10.987766] Adding 1048572k swap on /dev/vdb. Priority:-2 extents:1 across:1048572k FS [ OK ] Created slice User and Session Slice. [ OK ] Reached target Slices. [ OK ] Started Journal Service. [ OK ] Mounted Kernel Debug File System. [ OK ] Mounted POSIX Message Queue File System. [FAILED] Failed to start Remount Root and Kernel File Systems. See 'systemctl status systemd-remount-fs.service' for details. [ OK ] Started Create list of required sta…vice nodes for the current kernel. [ OK ] Activated swap /dev/disk/by-label/SWAP. [ OK ] Started Apply Kernel Variables. [ OK ] Mounted Huge Pages File System. [ OK ] Reached target Swap. Starting Configure read-only root support... Starting Create Static Device Nodes in /dev... Starting Flush Journal to Persistent Storage... [ OK ] Started Flush Journal to Persistent Storage. [ OK ] Started Create Static Device Nodes in /dev. [ OK ] Reached target Local File Systems (Pre). Mounting /home/green/git/lustre-release... Mounting /mnt... Starting udev Kernel Device Manager... [ OK ] Mounted /mnt. [ 11.238935] squashfs: version 4.0 (2009/01/31) Phillip Lougher [ OK ] Started udev Coldplug all Devices. [ OK ] Mounted /home/green/git/lustre-release. [ OK ] Started udev Kernel Device Manager. [ 11.425482] piix4_smbus 0000:00:01.3: SMBus Host Controller at 0x700, revision 0 [ 11.492550] input: PC Speaker as /devices/platform/pcspkr/input/input5 [ 11.569696] RAPL PMU: API unit is 2^-32 Joules, 0 fixed counters, 10737418240 ms ovfl timer [ 11.595360] EDAC sbridge: Ver: 1.1.2 [ 12.922114] Key type dns_resolver registered [ 13.190185] NFS: Registering the id_resolver key type [ 13.192058] Key type id_resolver registered [ 13.192849] Key type id_legacy registered [ OK ] Started Configure read-only root support. [ OK ] Reached target Local File Systems. Starting Rebuild Dynamic Linker Cache... Starting Mark the need to relabel after reboot... Starting Create Volatile Files and Directories... Starting Load/Save Random Seed... [ OK ] Started Mark the need to relabel after reboot. [ OK ] Started Load/Save Random Seed. [ OK ] Started Create Volatile Files and Directories. Starting RPC Bind... Starting Update UTMP about System Boot/Shutdown... [ OK ] Started Update UTMP about System Boot/Shutdown. [ OK ] Started RPC Bind. [ OK ] Started Rebuild Dynamic Linker Cache. Starting Update is Completed... [ OK ] Started Update is Completed. [ OK ] Reached target System Initialization. [ OK ] Started daily update of the root trust anchor for DNSSEC. [ OK ] Started Daily Cleanup of Temporary Directories. [ OK ] Listening on D-Bus System Message Bus Socket. [ OK ] Reached target Sockets. [ OK ] Started dnf makecache --timer. [ OK ] Reached target Timers. [ OK ] Reached target Basic System. [ OK ] Started irqbalance daemon. [ OK ] Started Hardware RNG Entropy Gatherer Daemon. Starting Login Service... [ OK ] Started D-Bus System Message Bus. Starting Restore /run/initramfs on shutdown... Starting Network Manager... [ OK ] Reached target sshd-keygen.target. [ OK ] Started Restore /run/initramfs on shutdown. [ OK ] Started Login Service. [ OK ] Started Network Manager. Starting Network Manager Wait Online... [ OK ] Reached target Network. Starting GSSAPI Proxy Daemon... Starting OpenSSH server daemon... Starting Dynamic System Tuning Daemon... Starting Hostname Service... [ OK ] Started OpenSSH server daemon. [ OK ] Started GSSAPI Proxy Daemon. [ OK ] Reached target NFS client services. [ OK ] Reached target Remote File Systems (Pre). [ OK ] Reached target Remote File Systems. Starting Permit User Sessions... [ OK ] Started Permit User Sessions. [ OK ] Started Getty on tty1. [ OK ] Started Serial Getty on ttyS1. [ OK ] Started Command Scheduler. [ OK ] Started Serial Getty on ttyS0. [ OK ] Reached target Login Prompts. [ OK ] Started Hostname Service. Starting Network Manager Script Dispatcher Service... [ OK ] Started Network Manager Script Dispatcher Service. [ OK ] Started Network Manager Wait Online. [ OK ] Reached target Network is Online. Starting System Logging Service... Starting Notify NFS peers of a restart... Starting Crash recovery kernel arming... [ OK ] Started Notify NFS peers of a restart. [ OK ] Started System Logging Service. Starting Authorization Manager... [ OK ] Started Authorization Manager. [ OK ] Started Dynamic System Tuning Daemon. [ OK ] Reached target Multi-User System. [ OK ] Reached target Graphical Interface. Starting Update UTMP about System Runlevel Changes... [ OK ] Started Update UTMP about System Runlevel Changes. [ OK ] Started Crash recovery kernel arming. Rocky Linux 8.10 (Green Obsidian) Kernel 4.18.0rh8.10-debug on an x86_64 oleg308-server login: [ 35.143358] libcfs: loading out-of-tree module taints kernel. [ 35.158926] Key type ._llcrypt registered [ 35.159898] Key type .llcrypt registered [ 35.196328] Lustre: DEBUG MARKER: oleg308-server.virtnet: executing set_hostid [ 42.509753] Lustre: DEBUG MARKER: oleg308-server.virtnet: executing load_modules_local [ 43.122975] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 43.128514] alg: No test for adler32 (adler32-zlib) [ 44.088354] Lustre: Lustre: Build Version: 2.17.0_RC4_2_g45eb11f [ 44.360200] LNet: Added LNI 192.168.203.108@tcp [8/256/0/180] [ 45.975231] Key type lgssc registered [ 46.633521] Lustre: Echo OBD driver; http://www.lustre.org/ [ 52.651712] ZFS: Loaded module v2.3.2-1, ZFS pool version 5000, ZFS filesystem version 5 [ 53.312893] LDISKFS-fs (vdc): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 56.537904] LDISKFS-fs (vdd): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 58.964064] LDISKFS-fs (vde): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 61.598040] LDISKFS-fs (vdf): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 66.628448] Lustre: DEBUG MARKER: oleg308-server.virtnet: executing load_modules_local [ 71.131702] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 71.151703] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 71.158421] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 72.249476] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 72.263972] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 72.305665] Lustre: lustre-MDT0000: new disk, initializing [ 72.331068] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 72.337881] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 73.654842] Lustre: DEBUG MARKER: oleg308-server.virtnet: executing set_default_debug vfstrace rpctrace dlmtrace neterror ha config ioctl super lfsck all [ 78.658651] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 78.681522] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 78.709355] Lustre: 6481:0:(mgs_llog.c:1348:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 78.723351] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [ 78.725021] Lustre: Skipped 1 previous similar message [ 78.762808] Lustre: lustre-MDT0001: new disk, initializing [ 78.783071] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 78.791087] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [ 78.794412] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 80.047319] Lustre: DEBUG MARKER: oleg308-server.virtnet: executing set_default_debug vfstrace rpctrace dlmtrace neterror ha config ioctl super lfsck all [ 82.247524] Lustre: Modifying parameter general.debug_raw_pointers=Y in log params [ 85.329526] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 85.353462] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 85.434521] Lustre: lustre-OST0000: new disk, initializing [ 85.437181] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 85.452732] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 87.255569] Lustre: DEBUG MARKER: oleg308-server.virtnet: executing set_default_debug vfstrace rpctrace dlmtrace neterror ha config ioctl super lfsck all [ 87.978685] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [ 87.984440] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 87.999266] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 92.439834] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 92.462721] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 92.493852] Lustre: lustre-OST0001: new disk, initializing [ 92.495434] Lustre: srv-lustre-OST0001: No data found on store. Initialize space. [ 92.510370] Lustre: lustre-OST0001: Imperative Recovery not enabled, recovery window 60-180 [ 93.226415] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x00000002c0000400-0x0000000300000400]:1:ost [ 93.230252] Lustre: cli-lustre-OST0001-super: Allocated super-sequence [0x00000002c0000400-0x0000000300000400]:1:ost] [ 93.251225] Lustre: lustre-OST0001-osc-MDT0000: update sequence from 0x100010000 to 0x2c0000401 [ 94.516472] Lustre: DEBUG MARKER: oleg308-server.virtnet: executing set_default_debug vfstrace rpctrace dlmtrace neterror ha config ioctl super lfsck all [ 100.811530] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 104.540867] Lustre: Setting parameter general.lod.*.mdt_hash=crush in log params [ 111.513768] Lustre: DEBUG MARKER: oleg308-server.virtnet: executing check_logdir /tmp/testlogs/ [ 113.189931] Lustre: DEBUG MARKER: oleg308-server.virtnet: executing yml_node [ 114.901680] Lustre: DEBUG MARKER: Client: 2.17.0.RC4 [ 116.198574] Lustre: DEBUG MARKER: MDS: 2.17.0.RC4 [ 117.454368] Lustre: DEBUG MARKER: OSS: 2.17.0.RC4 [ 118.067113] Lustre: DEBUG MARKER: -----============= acceptance-small: conf-sanity ============----- Fri Jan 2 15:18:17 EST 2026 [ 125.049390] Lustre: DEBUG MARKER: excepting tests: 102 106 115 32newtarball [ 125.780764] Lustre: DEBUG MARKER: skipping tests SLOW=no: 45 69 106 111 114 [ 139.232260] Lustre: lustre-MDT0000-lwp-OST0000: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 139.232985] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 139.237584] Lustre: Skipped 3 previous similar messages [ 139.239862] Lustre: Skipped 2 previous similar messages [ 143.614073] LustreError: 12182:0:(obd_class.h:479:obd_check_dev()) Device 28 not setup [ 143.642090] Lustre: server umount lustre-MDT0000 complete [ 145.151057] LustreError: 8388:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1767385125 with bad export cookie 6771873489386392030 [ 145.153614] LustreError: MGC192.168.203.108@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 145.157227] LustreError: 8388:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 149.471892] LustreError: 8397:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 149.472057] Lustre: lustre-MDT0001-lwp-OST0000: Connection to lustre-MDT0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 149.472593] Lustre: lustre-MDT0001: Not available for connect from 0@lo (stopping) [ 149.472602] Lustre: Skipped 1 previous similar message [ 149.481915] LustreError: 8397:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 6 previous similar messages [ 149.490099] Lustre: Skipped 1 previous similar message [ 150.495606] LustreError: 8397:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 150.495613] Lustre: lustre-MDT0001: Not available for connect from 0@lo (stopping) [ 150.505692] Lustre: Skipped 2 previous similar messages [ 151.309899] LustreError: 12384:0:(obd_class.h:479:obd_check_dev()) Device 27 not setup [ 151.314740] LustreError: 12384:0:(obd_class.h:479:obd_check_dev()) Skipped 7 previous similar messages [ 151.369422] Lustre: server umount lustre-MDT0001 complete [ 153.133168] LustreError: 12586:0:(obd_class.h:479:obd_check_dev()) Device 19 not setup [ 153.137619] LustreError: 12586:0:(obd_class.h:479:obd_check_dev()) Skipped 7 previous similar messages [ 153.165733] Lustre: server umount lustre-OST0000 complete [ 154.565405] Lustre: server umount lustre-OST0001 complete [ 156.583942] Lustre: DEBUG MARKER: oleg308-server.virtnet: executing set_hostid [ 160.862185] Lustre: DEBUG MARKER: oleg308-server.virtnet: executing load_modules_local [ 165.031360] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 167.654492] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 170.236466] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 173.352975] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 176.663601] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 176.702412] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 176.823168] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 176.836362] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 176.875249] Lustre: lustre-MDT0000: new disk, initializing [ 176.898175] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 176.903779] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 178.368138] Lustre: DEBUG MARKER: oleg308-server.virtnet: executing set_default_debug -1 all [ 183.613428] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 183.657976] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 183.722634] Lustre: 14967:0:(mgs_llog.c:1348:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 183.901235] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 185.456420] Lustre: DEBUG MARKER: oleg308-server.virtnet: executing set_default_debug -1 all [ 189.110565] Lustre: DEBUG MARKER: oleg308-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 190.978458] Lustre: DEBUG MARKER: oleg308-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 193.904780] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 193.939352] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 194.080161] Lustre: lustre-OST0000: new disk, initializing [ 194.083824] Lustre: Skipped 1 previous similar message [ 194.087124] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 194.089647] Lustre: Skipped 2 previous similar messages [ 194.129394] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 194.132610] Lustre: Skipped 1 previous similar message [ 196.026706] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [ 196.028987] Lustre: Skipped 1 previous similar message [ 196.030758] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 196.032466] Lustre: DEBUG MARKER: oleg308-server.virtnet: executing set_default_debug -1 all [ 196.066050] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 199.444439] Lustre: DEBUG MARKER: oleg308-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 200.858544] Lustre: DEBUG MARKER: oleg308-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 [ 200.957692] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [ 202.330390] Lustre: DEBUG MARKER: oleg308-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid 50 [ 202.394658] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [ 206.307035] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 206.307388] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 206.309747] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 206.311601] Lustre: Skipped 1 previous similar message [ 206.314266] LustreError: Skipped 1 previous similar message [ 209.132288] LustreError: 17906:0:(obd_class.h:479:obd_check_dev()) Device 19 not setup [ 209.134775] LustreError: 17906:0:(obd_class.h:479:obd_check_dev()) Skipped 9 previous similar messages [ 209.167174] Lustre: server umount lustre-OST0000 complete [ 211.424825] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 211.425544] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 211.427859] Lustre: Skipped 1 previous similar message [ 211.432120] Lustre: Skipped 2 previous similar messages [ 216.779763] LustreError: 18107:0:(obd_class.h:479:obd_check_dev()) Device 22 not setup [ 216.783303] LustreError: 18107:0:(obd_class.h:479:obd_check_dev()) Skipped 3 previous similar messages [ 216.815707] Lustre: server umount lustre-MDT0000 complete [ 218.171265] LustreError: 16715:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1767385198 with bad export cookie 6771873489386393549 [ 218.174425] LustreError: MGC192.168.203.108@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 218.177563] LustreError: 16715:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 218.281832] Lustre: server umount lustre-MDT0001 complete [ 219.707651] Lustre: DEBUG MARKER: == conf-sanity test 76a: set permanent params with lctl across mounts ========================================================== 15:19:59 (1767385199) [ 222.362398] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 222.560940] LustreError: 18835:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 222.569766] LustreError: 18835:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 1 previous similar message [ 222.594676] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 224.100097] Lustre: DEBUG MARKER: oleg308-server.virtnet: executing set_default_debug -1 all [ 227.810921] LustreError: 18835:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 227.830710] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 229.526976] Lustre: DEBUG MARKER: oleg308-server.virtnet: executing set_default_debug -1 all [ 231.977662] Lustre: DEBUG MARKER: oleg308-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 233.690505] Lustre: DEBUG MARKER: oleg308-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 236.975409] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 239.726775] Lustre: DEBUG MARKER: oleg308-server.virtnet: executing set_default_debug -1 all [ 242.343444] Lustre: DEBUG MARKER: oleg308-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 249.506578] Lustre: Modifying parameter general.osc.*.max_dirty_mb=452 in log params [ 252.897118] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 252.898315] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 252.901587] Lustre: Skipped 2 previous similar messages [ 252.906952] Lustre: Skipped 4 previous similar messages [ 258.957062] LustreError: 21137:0:(obd_class.h:479:obd_check_dev()) Device 9 not setup [ 258.961435] LustreError: 21137:0:(obd_class.h:479:obd_check_dev()) Skipped 13 previous similar messages [ 258.994774] Lustre: server umount lustre-MDT0000 complete [ 260.357435] LustreError: 18815:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1767385241 with bad export cookie 6771873489386394431 [ 260.361082] LustreError: 18815:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) Skipped 1 previous similar message [ 260.362353] LustreError: MGC192.168.203.108@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 260.480113] Lustre: server umount lustre-MDT0001 complete [ 272.174971] Lustre: server umount lustre-OST0000 complete [ 280.877553] Lustre: DEBUG MARKER: oleg308-server.virtnet: executing load_modules_local [ 284.685091] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 284.897152] LustreError: 22865:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 284.925251] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 284.928941] Lustre: Skipped 2 previous similar messages [ 286.301084] Lustre: DEBUG MARKER: oleg308-server.virtnet: executing set_default_debug -1 all [ 289.500713] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 290.947206] Lustre: DEBUG MARKER: oleg308-server.virtnet: executing set_default_debug -1 all [ 292.348517] Lustre: Setting parameter general.debug_raw_pointers=Y in log params [ 295.402670] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 297.754753] Lustre: DEBUG MARKER: oleg308-server.virtnet: executing set_default_debug -1 all [ 300.857557] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 300.885582] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 300.971933] Lustre: lustre-OST0001: new disk, initializing [ 300.978011] Lustre: srv-lustre-OST0001: No data found on store. Initialize space. [ 302.238824] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x00000002c0000400-0x0000000300000400]:1:ost [ 302.246119] Lustre: cli-lustre-OST0001-super: Allocated super-sequence [0x00000002c0000400-0x0000000300000400]:1:ost] [ 302.301921] Lustre: lustre-OST0001-osc-MDT0000: update sequence from 0x100010000 to 0x2c0000401 [ 303.610886] Lustre: DEBUG MARKER: oleg308-server.virtnet: executing set_default_debug -1 all [ 308.789798] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 310.533745] Lustre: Setting parameter general.lod.*.mdt_hash=crush in log params [ 320.481180] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 320.488211] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 320.495474] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 320.497956] Lustre: Skipped 3 previous similar messages [ 326.612233] LustreError: 26346:0:(obd_class.h:479:obd_check_dev()) Device 28 not setup [ 326.615716] LustreError: 26346:0:(obd_class.h:479:obd_check_dev()) Skipped 17 previous similar messages [ 326.653330] Lustre: server umount lustre-MDT0000 complete [ 328.286369] LustreError: 22844:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1767385308 with bad export cookie 6771873489386395278 [ 328.293781] LustreError: MGC192.168.203.108@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 328.296441] LustreError: 22844:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 328.431341] Lustre: server umount lustre-MDT0001 complete [ 339.403123] Lustre: server umount lustre-OST0000 complete [ 351.353494] Lustre: server umount lustre-OST0001 complete [ 357.577787] Lustre: DEBUG MARKER: oleg308-server.virtnet: executing load_modules_local [ 361.801507] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 361.982608] LustreError: 28125:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 362.002933] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 362.005601] Lustre: Skipped 3 previous similar messages [ 363.498139] Lustre: DEBUG MARKER: oleg308-server.virtnet: executing set_default_debug -1 all [ 366.876947] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 368.636595] Lustre: DEBUG MARKER: oleg308-server.virtnet: executing set_default_debug -1 all [ 370.047357] Lustre: 29241:0:(mgs_llog.c:1348:mgs_modify_param()) MGS: modify general/debug_raw_pointers=Y (mode = 0) failed: rc = -17 [ 373.014629] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 375.200773] Lustre: DEBUG MARKER: oleg308-server.virtnet: executing set_default_debug -1 all [ 379.259264] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 381.706361] Lustre: DEBUG MARKER: oleg308-server.virtnet: executing set_default_debug -1 all [ 391.161270] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 392.762676] Lustre: 31097:0:(mgs_llog.c:1348:mgs_modify_param()) MGS: modify general/lod.*.mdt_hash=crush (mode = 0) failed: rc = -17 [ 402.913908] Lustre: lustre-MDT0000-lwp-OST0000: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 402.916898] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 402.918052] Lustre: Skipped 4 previous similar messages [ 402.919205] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 402.926193] Lustre: Skipped 7 previous similar messages [ 405.950441] LustreError: 31298:0:(obd_class.h:479:obd_check_dev()) Device 10 not setup [ 405.952894] LustreError: 31298:0:(obd_class.h:479:obd_check_dev()) Skipped 25 previous similar messages [ 405.978497] Lustre: server umount lustre-MDT0000 complete [ 407.708814] LustreError: 29623:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1767385388 with bad export cookie 6771873489386396573 [ 407.718425] LustreError: MGC192.168.203.108@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 407.853268] Lustre: server umount lustre-MDT0001 complete [ 420.031084] Lustre: server umount lustre-OST0000 complete [ 430.482458] Lustre: server umount lustre-OST0001 complete [ 434.228836] Lustre: DEBUG MARKER: == conf-sanity test 76b: verify params log setup correctly ========================================================== 15:23:33 (1767385413) [ 444.868855] Lustre: DEBUG MARKER: oleg308-server.virtnet: executing load_modules_local [ 448.982346] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 449.182764] LustreError: 33876:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 449.195230] LustreError: 33876:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 1 previous similar message [ 450.627899] Lustre: DEBUG MARKER: oleg308-server.virtnet: executing set_default_debug -1 all [ 453.671868] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 455.463503] Lustre: DEBUG MARKER: oleg308-server.virtnet: executing set_default_debug -1 all [ 456.820479] Lustre: 34995:0:(mgs_llog.c:1348:mgs_modify_param()) MGS: modify general/debug_raw_pointers=Y (mode = 0) failed: rc = -17 [ 459.369068] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 461.677638] Lustre: DEBUG MARKER: oleg308-server.virtnet: executing set_default_debug -1 all [ 465.021151] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 467.036501] Lustre: DEBUG MARKER: oleg308-server.virtnet: executing set_default_debug -1 all [ 481.031129] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 482.627313] Lustre: 36869:0:(mgs_llog.c:1348:mgs_modify_param()) MGS: modify general/lod.*.mdt_hash=crush (mode = 0) failed: rc = -17 [ 489.954426] Lustre: lustre-MDT0000-lwp-OST0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 489.958267] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 489.958611] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 489.959624] Lustre: Skipped 5 previous similar messages [ 495.176913] LustreError: 37021:0:(obd_class.h:479:obd_check_dev()) Device 10 not setup [ 495.178822] LustreError: 37021:0:(obd_class.h:479:obd_check_dev()) Skipped 25 previous similar messages [ 495.210247] Lustre: server umount lustre-MDT0000 complete [ 496.821280] LustreError: 33857:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1767385477 with bad export cookie 6771873489386397679 [ 496.827618] LustreError: MGC192.168.203.108@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 496.828062] LustreError: 33857:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) Skipped 6 previous similar messages [ 496.979410] Lustre: server umount lustre-MDT0001 complete [ 508.414480] Lustre: server umount lustre-OST0000 complete [ 520.245898] Lustre: server umount lustre-OST0001 complete [ 523.330967] Lustre: DEBUG MARKER: == conf-sanity test 76c: verify changelog_mask is applied with lctl set_param -P ========================================================== 15:25:02 (1767385502) [ 529.803775] Lustre: DEBUG MARKER: oleg308-server.virtnet: executing load_modules_local [ 534.078446] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 534.243861] LustreError: 38991:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 534.253513] LustreError: 38991:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 1 previous similar message [ 534.271726] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 534.274336] Lustre: Skipped 7 previous similar messages [ 535.799170] Lustre: DEBUG MARKER: oleg308-server.virtnet: executing set_default_debug -1 all [ 539.994914] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 541.883146] Lustre: DEBUG MARKER: oleg308-server.virtnet: executing set_default_debug -1 all [ 543.286159] Lustre: 40108:0:(mgs_llog.c:1348:mgs_modify_param()) MGS: modify general/debug_raw_pointers=Y (mode = 0) failed: rc = -17 [ 546.465736] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 549.199070] Lustre: DEBUG MARKER: oleg308-server.virtnet: executing set_default_debug -1 all [ 553.332732] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 556.310846] Lustre: DEBUG MARKER: oleg308-server.virtnet: executing set_default_debug -1 all [ 565.784800] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 573.011734] Lustre: Setting parameter general.mdd.*.changelog_mask=-CLOSE in log params [ 573.016605] Lustre: Skipped 1 previous similar message [ 574.949818] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 574.957206] Lustre: Skipped 2 previous similar messages [ 580.682260] Lustre: server umount lustre-MDT0000 complete [ 582.494540] LustreError: 41965:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1767385563 with bad export cookie 6771873489386398785 [ 582.502546] LustreError: MGC192.168.203.108@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 582.509090] LustreError: 41965:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 588.730268] Lustre: server umount lustre-MDT0001 complete [ 591.842047] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 592.356838] LustreError: 3624:0:(client.c:1390:ptlrpc_import_delay_req()) @@@ invalidate in flight req@ffff94a8847da680 x1853237520293504/t0(0) o250->MGC192.168.203.108@tcp@0@lo:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl Rpc:NQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 [ 594.228857] Lustre: DEBUG MARKER: oleg308-server.virtnet: executing set_default_debug -1 all [ 597.790373] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 599.455633] Lustre: DEBUG MARKER: oleg308-server.virtnet: executing set_default_debug -1 all [ 601.945812] Lustre: DEBUG MARKER: oleg308-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 604.860747] Lustre: DEBUG MARKER: mdc.lustre-MDT0000-mdc-*.mds_server_uuid in FULL state after 2 sec [ 606.489873] Lustre: DEBUG MARKER: oleg308-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 607.213404] Lustre: DEBUG MARKER: mdc.lustre-MDT0001-mdc-*.mds_server_uuid in FULL state after 0 sec [ 612.648331] Lustre: lustre-MDT0001-lwp-OST0000: Connection restored to 0@lo (at 0@lo) [ 614.704673] Lustre: server umount lustre-MDT0000 complete [ 616.313608] LustreError: 38971:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1767385596 with bad export cookie 6771873489386399926 [ 616.320750] LustreError: 38971:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 616.321332] LustreError: MGC192.168.203.108@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 616.459186] Lustre: server umount lustre-MDT0001 complete [ 627.453132] LustreError: 44761:0:(obd_class.h:479:obd_check_dev()) Device 23 not setup [ 627.457014] LustreError: 44761:0:(obd_class.h:479:obd_check_dev()) Skipped 57 previous similar messages [ 627.478890] Lustre: server umount lustre-OST0000 complete [ 638.536264] Lustre: server umount lustre-OST0001 complete [ 642.569656] Lustre: DEBUG MARKER: == conf-sanity test 76d: verify llite.*.xattr_cache can be set by 'lctl set_param -P' correctly ========================================================== 15:27:01 (1767385621) [ 649.136728] Lustre: DEBUG MARKER: oleg308-server.virtnet: executing load_modules_local [ 653.351403] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 653.572191] LustreError: 46331:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 653.581827] LustreError: 46331:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 10 previous similar messages [ 654.762076] Lustre: DEBUG MARKER: oleg308-server.virtnet: executing set_default_debug -1 all [ 658.654462] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 660.125538] Lustre: DEBUG MARKER: oleg308-server.virtnet: executing set_default_debug -1 all [ 661.057138] Lustre: 47451:0:(mgs_llog.c:1348:mgs_modify_param()) MGS: modify general/debug_raw_pointers=Y (mode = 0) failed: rc = -17 [ 661.059485] Lustre: 47451:0:(mgs_llog.c:1348:mgs_modify_param()) Skipped 1 previous similar message [ 664.107082] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 666.845775] Lustre: DEBUG MARKER: oleg308-server.virtnet: executing set_default_debug -1 all [ 670.867548] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 673.536502] Lustre: DEBUG MARKER: oleg308-server.virtnet: executing set_default_debug -1 all [ 682.762570] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 689.964238] Lustre: Setting parameter general.llite.*.xattr_cache=0 in log params [ 694.759308] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 694.761539] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 694.763503] Lustre: Skipped 27 previous similar messages [ 700.854349] Lustre: server umount lustre-MDT0000 complete [ 702.614911] LustreError: 47838:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1767385683 with bad export cookie 6771873489386400738 [ 702.621673] LustreError: 47838:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 702.622341] LustreError: MGC192.168.203.108@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 702.784618] Lustre: server umount lustre-MDT0001 complete [ 714.174803] Lustre: server umount lustre-OST0000 complete [ 726.057925] Lustre: server umount lustre-OST0001 complete [ 730.123585] Lustre: DEBUG MARKER: == conf-sanity test 77: comma-separated MGS NIDs and failover node NIDs ========================================================== 15:28:29 (1767385709) [ 731.336965] Lustre: DEBUG MARKER: SKIP: conf-sanity test_77 mixed loopback and real device not working [ 732.176453] Lustre: DEBUG MARKER: == conf-sanity test 78: run resize2fs on MDT and OST filesystems ========================================================== 15:28:31 (1767385711) [ 736.683447] Lustre: DEBUG MARKER: oleg308-server.virtnet: executing set_hostid [ 740.391150] Lustre: DEBUG MARKER: oleg308-server.virtnet: executing load_modules_local [ 744.610970] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 747.473330] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 750.133043] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 750.165307] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 750.312327] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 750.371242] Lustre: lustre-MDT0000: new disk, initializing [ 750.420802] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 752.170368] Lustre: DEBUG MARKER: oleg308-server.virtnet: executing set_default_debug -1 all [ 756.047580] Lustre: DEBUG MARKER: oleg308-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 759.195634] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 759.228205] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 760.537705] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:0:ost] [ 760.550771] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x240000400 [ 761.815925] Lustre: DEBUG MARKER: oleg308-server.virtnet: executing set_default_debug -1 all [ 765.461733] Lustre: DEBUG MARKER: oleg308-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 766.437786] Lustre: DEBUG MARKER: create test files [ 781.281041] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 781.283258] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 781.286506] Lustre: Skipped 9 previous similar messages [ 783.259295] Lustre: server umount lustre-OST0000 complete [ 784.784189] Lustre: server umount lustre-MDT0000 complete [ 788.851476] Lustre: DEBUG MARKER: oleg308-server.virtnet: executing unload_modules_local [ 790.038464] Key type lgssc unregistered [ 790.163069] LNet: 54832:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 790.165425] LNetError: 54832:0:(acceptor.c:264:lnet_acceptor_remove_socket()) Interface ens2 not found [ 790.175503] LNet: Removed LNI 192.168.203.108@tcp [ 790.536090] Key type .llcrypt unregistered [ 790.537433] Key type ._llcrypt unregistered [ 801.389636] Key type ._llcrypt registered [ 801.390749] Key type .llcrypt registered [ 801.439386] Lustre: DEBUG MARKER: oleg308-server.virtnet: executing load_modules_local [ 801.783575] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 801.828128] alg: No test for adler32 (adler32-zlib) [ 802.684810] Lustre: Lustre: Build Version: 2.17.0_RC4_2_g45eb11f [ 802.768856] LNet: Added LNI 192.168.203.108@tcp [8/256/0/180] [ 804.343160] Key type lgssc registered [ 804.877896] Lustre: Echo OBD driver; http://www.lustre.org/ [ 807.960503] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 807.971114] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 809.157187] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 810.887228] Lustre: DEBUG MARKER: oleg308-server.virtnet: executing set_default_debug -1 all [ 813.397088] Lustre: DEBUG MARKER: oleg308-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 816.261207] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 816.357918] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 818.059324] Lustre: DEBUG MARKER: oleg308-server.virtnet: executing set_default_debug -1 all [ 818.409687] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x240000400:102 to 0x240000400:129) [ 818.457899] Lustre: 57412:0:(osd_internal.h:1459:osd_trans_exec_op()) lustre-OST0000: opcode 8: before 515 < left 544, rollback = 9 [ 818.463458] Lustre: 57412:0:(osd_handler.c:2096:osd_trans_dump_creds()) create: 32/128/0, destroy: 0/0/0 [ 818.468071] Lustre: 57412:0:(osd_handler.c:2103:osd_trans_dump_creds()) attr_set: 0/0/0, xattr_set: 0/0/0 [ 818.475441] Lustre: 57412:0:(osd_handler.c:2113:osd_trans_dump_creds()) write: 2/13/1, punch: 0/0/0, quota 1/3/0 [ 818.482897] Lustre: 57412:0:(osd_handler.c:2120:osd_trans_dump_creds()) insert: 32/544/0, delete: 0/0/0 [ 818.487506] Lustre: 57412:0:(osd_handler.c:2127:osd_trans_dump_creds()) ref_add: 0/0/0, ref_del: 0/0/0 [ 820.524911] Lustre: DEBUG MARKER: oleg308-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 823.753889] Lustre: DEBUG MARKER: check files after expanding the MDT and OST filesystems [ 825.786204] Lustre: DEBUG MARKER: create more files after expanding the MDT and OST filesystems [ 828.897819] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 828.897867] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 828.903956] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 833.971157] LustreError: 58154:0:(obd_class.h:479:obd_check_dev()) Device 13 not setup [ 834.031277] Lustre: server umount lustre-OST0000 complete [ 835.953886] LustreError: 58357:0:(obd_class.h:479:obd_check_dev()) Device 8 not setup [ 835.956959] LustreError: 58357:0:(obd_class.h:479:obd_check_dev()) Skipped 1 previous similar message [ 836.021594] Lustre: server umount lustre-MDT0000 complete [ 840.612939] Lustre: DEBUG MARKER: oleg308-server.virtnet: executing unload_modules_local [ 841.950558] Key type lgssc unregistered [ 842.103759] LNet: 58887:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 842.110229] LNetError: 58887:0:(acceptor.c:264:lnet_acceptor_remove_socket()) Interface ens2 not found [ 842.123597] LNet: Removed LNI 192.168.203.108@tcp [ 842.506283] Key type .llcrypt unregistered [ 842.507856] Key type ._llcrypt unregistered [ 856.082265] Key type ._llcrypt registered [ 856.083186] Key type .llcrypt registered [ 856.114210] Lustre: DEBUG MARKER: oleg308-server.virtnet: executing load_modules_local [ 856.399767] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 856.447142] alg: No test for adler32 (adler32-zlib) [ 857.322862] Lustre: Lustre: Build Version: 2.17.0_RC4_2_g45eb11f [ 857.425961] LNet: Added LNI 192.168.203.108@tcp [8/256/0/180] [ 859.015130] Key type lgssc registered [ 859.589241] Lustre: Echo OBD driver; http://www.lustre.org/ [ 862.079831] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 862.086603] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 863.257177] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 864.836256] Lustre: DEBUG MARKER: oleg308-server.virtnet: executing set_default_debug -1 all [ 867.205322] Lustre: DEBUG MARKER: oleg308-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 869.739401] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 869.895695] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 872.433091] Lustre: DEBUG MARKER: oleg308-server.virtnet: executing set_default_debug -1 all [ 872.999186] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x240000400:140 to 0x240000400:161) [ 873.035199] Lustre: 61565:0:(osd_internal.h:1459:osd_trans_exec_op()) lustre-OST0000: opcode 8: before 515 < left 544, rollback = 9 [ 873.039299] Lustre: 61565:0:(osd_handler.c:2096:osd_trans_dump_creds()) create: 32/128/0, destroy: 0/0/0 [ 873.042632] Lustre: 61565:0:(osd_handler.c:2103:osd_trans_dump_creds()) attr_set: 0/0/0, xattr_set: 0/0/0 [ 873.045865] Lustre: 61565:0:(osd_handler.c:2113:osd_trans_dump_creds()) write: 2/13/1, punch: 0/0/0, quota 1/3/0 [ 873.049141] Lustre: 61565:0:(osd_handler.c:2120:osd_trans_dump_creds()) insert: 32/544/0, delete: 0/0/0 [ 873.052237] Lustre: 61565:0:(osd_handler.c:2127:osd_trans_dump_creds()) ref_add: 0/0/0, ref_del: 0/0/0 [ 874.546703] Lustre: DEBUG MARKER: oleg308-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 877.849761] Lustre: DEBUG MARKER: check files after shrinking the MDT and OST filesystems [ 883.170765] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 883.170894] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 883.185149] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 887.094151] LustreError: 62258:0:(obd_class.h:479:obd_check_dev()) Device 13 not setup [ 887.146664] Lustre: server umount lustre-OST0000 complete [ 889.057942] LustreError: 62460:0:(obd_class.h:479:obd_check_dev()) Device 8 not setup [ 889.061092] LustreError: 62460:0:(obd_class.h:479:obd_check_dev()) Skipped 1 previous similar message [ 889.118503] Lustre: server umount lustre-MDT0000 complete [ 894.132406] Lustre: DEBUG MARKER: oleg308-server.virtnet: executing unload_modules_local [ 895.634651] Key type lgssc unregistered [ 895.804529] LNet: 62991:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 895.809491] LNetError: 62991:0:(acceptor.c:264:lnet_acceptor_remove_socket()) Interface ens2 not found [ 895.825630] LNet: Removed LNI 192.168.203.108@tcp [ 896.282512] Key type .llcrypt unregistered [ 896.284657] Key type ._llcrypt unregistered [ 903.315037] Key type ._llcrypt registered [ 903.316429] Key type .llcrypt registered [ 903.366601] Lustre: DEBUG MARKER: oleg308-server.virtnet: executing set_hostid [ 910.342637] Lustre: DEBUG MARKER: oleg308-server.virtnet: executing load_modules_local [ 910.621355] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 910.627251] alg: No test for adler32 (adler32-zlib) [ 911.502008] Lustre: Lustre: Build Version: 2.17.0_RC4_2_g45eb11f [ 911.610061] LNet: Added LNI 192.168.203.108@tcp [8/256/0/180] [ 913.199202] Key type lgssc registered [ 913.788621] Lustre: Echo OBD driver; http://www.lustre.org/ [ 918.040387] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 921.334364] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 924.452661] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 927.160243] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 930.437498] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 930.478753] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 930.489825] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 931.623064] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 931.642672] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 931.704893] Lustre: lustre-MDT0000: new disk, initializing [ 931.745243] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 931.754578] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 933.632771] Lustre: DEBUG MARKER: oleg308-server.virtnet: executing set_default_debug -1 all [ 938.473587] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 938.499642] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 938.528266] Lustre: 66094:0:(mgs_llog.c:1348:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 938.551572] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [ 938.553069] Lustre: Skipped 1 previous similar message [ 938.586773] Lustre: lustre-MDT0001: new disk, initializing [ 938.619968] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 938.630967] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [ 938.634127] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 939.751604] Lustre: DEBUG MARKER: oleg308-server.virtnet: executing set_default_debug -1 all [ 943.426909] Lustre: DEBUG MARKER: oleg308-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 945.114557] Lustre: DEBUG MARKER: oleg308-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 947.268048] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 947.291653] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 947.377500] Lustre: lustre-OST0000: new disk, initializing [ 947.379680] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 947.406394] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 949.336926] Lustre: DEBUG MARKER: oleg308-server.virtnet: executing set_default_debug -1 all [ 950.473356] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [ 950.480344] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 950.534932] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 952.887731] Lustre: DEBUG MARKER: oleg308-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 954.785530] Lustre: DEBUG MARKER: oleg308-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 [ 954.910186] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [ 956.692050] Lustre: DEBUG MARKER: oleg308-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid 50 [ 956.772876] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [ 960.994147] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 960.994444] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 960.999679] LustreError: Skipped 1 previous similar message [ 961.009338] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 961.015583] Lustre: Skipped 1 previous similar message [ 963.761203] LustreError: 69032:0:(obd_class.h:479:obd_check_dev()) Device 19 not setup [ 963.817876] Lustre: server umount lustre-OST0000 complete [ 966.114479] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 966.115443] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 966.121705] Lustre: Skipped 2 previous similar messages [ 966.128252] Lustre: Skipped 1 previous similar message [ 971.233161] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 971.238504] Lustre: Skipped 1 previous similar message [ 971.793080] LustreError: 69237:0:(obd_class.h:479:obd_check_dev()) Device 22 not setup [ 971.797411] LustreError: 69237:0:(obd_class.h:479:obd_check_dev()) Skipped 3 previous similar messages [ 971.834304] Lustre: server umount lustre-MDT0000 complete [ 973.673166] LustreError: 66086:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1767385954 with bad export cookie 17080076598406144814 [ 973.679878] LustreError: MGC192.168.203.108@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 973.680687] LustreError: 66086:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 973.769758] LustreError: 69439:0:(obd_class.h:479:obd_check_dev()) Device 21 not setup [ 973.774413] LustreError: 69439:0:(obd_class.h:479:obd_check_dev()) Skipped 5 previous similar messages [ 973.837562] Lustre: server umount lustre-MDT0001 complete [ 977.833564] Lustre: DEBUG MARKER: == conf-sanity test 79: format MDT/OST without mgs option (should return errors) ========================================================== 15:32:37 (1767385957) [ 982.714050] Lustre: DEBUG MARKER: oleg308-server.virtnet: executing load_modules_local [ 989.070057] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 1001.133206] Lustre: DEBUG MARKER: oleg308-server.virtnet: executing set_hostid [ 1005.537541] Lustre: DEBUG MARKER: oleg308-server.virtnet: executing load_modules_local [ 1010.274124] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 1013.463760] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 1016.257150] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 1019.387251] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 1022.447379] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 1022.479935] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1022.610294] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 1022.626846] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 1022.678265] Lustre: lustre-MDT0000: new disk, initializing [ 1022.711480] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 1022.720508] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 1024.222959] Lustre: DEBUG MARKER: oleg308-server.virtnet: executing set_default_debug -1 all [ 1029.989397] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 1030.021312] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 1030.061703] Lustre: 73670:0:(mgs_llog.c:1348:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 1030.100484] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [ 1030.104967] Lustre: Skipped 1 previous similar message [ 1030.166827] Lustre: lustre-MDT0001: new disk, initializing [ 1030.208373] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 1030.225143] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [ 1030.229397] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 1031.997820] Lustre: DEBUG MARKER: oleg308-server.virtnet: executing set_default_debug -1 all [ 1035.829787] Lustre: DEBUG MARKER: oleg308-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 1037.434513] Lustre: DEBUG MARKER: oleg308-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 1040.495264] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 1040.536340] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 1040.656613] Lustre: lustre-OST0000: new disk, initializing [ 1040.658296] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 1040.697326] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 1041.939852] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [ 1041.946548] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 1042.001156] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 1043.244732] Lustre: DEBUG MARKER: oleg308-server.virtnet: executing set_default_debug -1 all [ 1047.249874] Lustre: DEBUG MARKER: oleg308-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 1049.054033] Lustre: DEBUG MARKER: oleg308-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 [ 1049.161946] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [ 1050.885963] Lustre: DEBUG MARKER: oleg308-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid 50 [ 1050.952701] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [ 1052.130987] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 1052.134997] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 1052.136895] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 1052.146600] Lustre: Skipped 1 previous similar message [ 1052.153844] Lustre: Skipped 1 previous similar message [ 1057.251260] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 1057.256668] Lustre: Skipped 1 previous similar message [ 1057.840183] LustreError: 76609:0:(obd_class.h:479:obd_check_dev()) Device 19 not setup [ 1057.845749] LustreError: 76609:0:(obd_class.h:479:obd_check_dev()) Skipped 7 previous similar messages [ 1057.905430] Lustre: server umount lustre-OST0000 complete [ 1060.832502] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 1060.837034] LustreError: Skipped 1 previous similar message [ 1060.839687] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 1067.488793] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 1067.491519] Lustre: Skipped 4 previous similar messages [ 1074.143130] Lustre: lustre-MDT0000 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 1074.204663] LustreError: 76811:0:(obd_class.h:479:obd_check_dev()) Device 22 not setup [ 1074.209608] LustreError: 76811:0:(obd_class.h:479:obd_check_dev()) Skipped 3 previous similar messages [ 1074.261499] Lustre: server umount lustre-MDT0000 complete [ 1076.161836] LustreError: 73661:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1767386056 with bad export cookie 17080076598406145682 [ 1076.173185] LustreError: MGC192.168.203.108@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 1076.356269] Lustre: server umount lustre-MDT0001 complete [ 1080.442521] Lustre: DEBUG MARKER: == conf-sanity test 80: mgc import reconnect race ======== 15:34:19 (1767386059) [ 1083.418035] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1083.589982] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 1085.283854] Lustre: DEBUG MARKER: oleg308-server.virtnet: executing set_default_debug -1 all [ 1088.900923] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 1088.992336] Lustre: lustre-MDT0001: Not available for connect from 0@lo (not set up) [ 1088.996127] Lustre: Skipped 2 previous similar messages [ 1090.951957] Lustre: DEBUG MARKER: oleg308-server.virtnet: executing set_default_debug -1 all [ 1093.539735] Lustre: DEBUG MARKER: oleg308-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 1095.297602] Lustre: DEBUG MARKER: oleg308-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 1098.205339] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 1101.120074] Lustre: DEBUG MARKER: oleg308-server.virtnet: executing set_default_debug -1 all [ 1103.345431] Lustre: DEBUG MARKER: oleg308-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 1105.221736] Lustre: 80021:0:(genops.c:1791:obd_export_evict_by_uuid()) MGS: evicting 48d1b225-7b2b-4e74-9981-1a29c4cd8d79 at adminstrative request [ 1109.671097] LustreError: MGC192.168.203.108@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 1109.681424] Lustre: *** cfs_fail_loc=906, val=2147483648*** [ 1120.231227] Lustre: MGS: Client 48d1b225-7b2b-4e74-9981-1a29c4cd8d79 (at 0@lo) reconnecting [ 1120.238950] Lustre: Evicted from MGS (at 0@lo) after server handle changed from 0xed089f2c1075ea04 to 0xed089f2c1075ec96 [ 1120.249190] Lustre: MGC192.168.203.108@tcp: Connection restored to 0@lo (at 0@lo) [ 1138.756841] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 1138.788580] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 1138.879679] Lustre: lustre-OST0001: new disk, initializing [ 1138.881591] Lustre: srv-lustre-OST0001: No data found on store. Initialize space. [ 1138.939799] Lustre: lustre-OST0001: Imperative Recovery not enabled, recovery window 60-180 [ 1138.943746] Lustre: Skipped 2 previous similar messages [ 1141.538338] Lustre: DEBUG MARKER: oleg308-server.virtnet: executing set_default_debug -1 all [ 1145.481424] Lustre: DEBUG MARKER: oleg308-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0001-osc-[-0-9a-f]*.ost_server_uuid [ 1146.977235] LustreError: 81149:0:(obd_class.h:479:obd_check_dev()) Device 25 not setup [ 1146.981271] LustreError: 81149:0:(obd_class.h:479:obd_check_dev()) Skipped 13 previous similar messages [ 1147.008495] Lustre: server umount lustre-OST0001 complete [ 1147.919103] LustreError: 79279:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 1147.925824] LustreError: 79279:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 2 previous similar messages [ 1150.945778] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 1150.945991] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 1150.949778] LustreError: Skipped 1 previous similar message [ 1150.957164] Lustre: Skipped 1 previous similar message [ 1150.960637] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 1150.965611] Lustre: Skipped 1 previous similar message [ 1152.995845] LustreError: 79280:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 1153.004140] LustreError: 79280:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 2 previous similar messages [ 1154.864831] Lustre: server umount lustre-OST0000 complete [ 1162.804281] Lustre: server umount lustre-MDT0000 complete [ 1163.234258] LustreError: 77684:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 1163.247091] LustreError: 77684:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 1 previous similar message [ 1164.589278] LustreError: 77666:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1767386145 with bad export cookie 17080076598406147222 [ 1164.596967] LustreError: MGC192.168.203.108@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 1164.600331] LustreError: 77666:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) Skipped 8 previous similar messages [ 1164.687648] LustreError: 81763:0:(obd_class.h:479:obd_check_dev()) Device 23 not setup [ 1164.690536] LustreError: 81763:0:(obd_class.h:479:obd_check_dev()) Skipped 15 previous similar messages [ 1164.743486] Lustre: server umount lustre-MDT0001 complete [ 1168.872560] Lustre: DEBUG MARKER: == conf-sanity test 81: sparse OST indexing ============== 15:35:48 (1767386148) [ 1169.682671] Lustre: DEBUG MARKER: SKIP: conf-sanity test_81 needs >= 3 OSTs [ 1170.635469] Lustre: DEBUG MARKER: == conf-sanity test 82a: specify OSTs for file (succeed) or directory (succeed) ========================================================== 15:35:49 (1767386149) [ 1171.414750] Lustre: DEBUG MARKER: SKIP: conf-sanity test_82a needs >= 3 OSTs [ 1172.263179] Lustre: DEBUG MARKER: == conf-sanity test 82b: specify OSTs for file with --pool and --ost-list options ========================================================== 15:35:51 (1767386151) [ 1172.951709] Lustre: DEBUG MARKER: SKIP: conf-sanity test_82b needs >= 4 OSTs [ 1173.527518] Lustre: DEBUG MARKER: == conf-sanity test 82c: specify sparse OSTs for setstripe (should not crash) ========================================================== 15:35:52 (1767386152) [ 1174.253811] Lustre: DEBUG MARKER: SKIP: conf-sanity test_82c needs >= 4 OSTs [ 1175.153848] Lustre: DEBUG MARKER: == conf-sanity test 83: ENOSPACE on OST doesn't cause message VFS: Busy inodes after unmount ... ========================================================== 15:35:54 (1767386154) [ 1175.921088] Lustre: DEBUG MARKER: mount the OST /dev/mapper/ost1_flakey as a ldiskfs filesystem [ 1178.348597] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 1178.790421] loop: module loaded [ 1178.796803] loop0: detected capacity change from 0 to 4294967296 [ 1178.812638] LDISKFS-fs (loop0): mounted filesystem with ordered data mode. Opts: (null) [ 1179.541902] Lustre: DEBUG MARKER: run llverfs in partial mode on the OST ldiskfs /mnt/lustre-ost1 [ 1180.625539] Lustre: DEBUG MARKER: oleg308-server.virtnet: executing run_llverfs /mnt/lustre-ost1 -vpl no [ 1182.085736] Lustre: DEBUG MARKER: unmount the OST /dev/mapper/ost1_flakey [ 1185.479016] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 1186.754804] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 1188.216682] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 1189.551634] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 1192.070326] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 1192.099558] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 1192.112348] LustreError: 84427:0:(obd_config.c:783:class_setup()) setup lustre-OST0000-osd failed (-28) [ 1192.115471] LustreError: 84427:0:(obd_mount.c:193:lustre_start_simple()) lustre-OST0000-osd setup error -28 [ 1192.118651] LustreError: 84427:0:(tgt_mount.c:2540:server_fill_super()) Unable to start osd on /dev/mapper/ost1_flakey: -28 [ 1192.122373] LustreError: 84427:0:(super25.c:199:lustre_fill_super()) llite: Unable to mount : rc = -28 [ 1200.488286] Lustre: DEBUG MARKER: oleg308-server.virtnet: executing set_hostid [ 1204.862454] Lustre: DEBUG MARKER: oleg308-server.virtnet: executing load_modules_local [ 1209.320959] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 1212.494644] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 1215.466924] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 1218.816481] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 1222.112686] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 1222.152577] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1222.269912] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 1222.290444] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 1222.347608] Lustre: lustre-MDT0000: new disk, initializing [ 1222.381490] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 1222.393172] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 1223.981086] Lustre: DEBUG MARKER: oleg308-server.virtnet: executing set_default_debug -1 all [ 1229.108160] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 1229.133384] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 1229.180293] Lustre: 87266:0:(mgs_llog.c:1348:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 1229.316373] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 1230.796182] Lustre: DEBUG MARKER: oleg308-server.virtnet: executing set_default_debug -1 all [ 1234.756432] Lustre: DEBUG MARKER: oleg308-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 1236.384126] Lustre: DEBUG MARKER: oleg308-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 1239.408982] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 1239.448377] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 1241.752193] Lustre: DEBUG MARKER: oleg308-server.virtnet: executing set_default_debug -1 all [ 1245.488463] Lustre: DEBUG MARKER: oleg308-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 1246.722858] Lustre: DEBUG MARKER: oleg308-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 [ 1247.742610] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 1247.785937] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 1248.873140] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 2 sec [ 1250.079882] Lustre: DEBUG MARKER: oleg308-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid 50 [ 1250.140051] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [ 1252.835229] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 1252.837380] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 1252.840126] LustreError: Skipped 1 previous similar message [ 1252.847253] Lustre: Skipped 2 previous similar messages [ 1252.848282] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 1252.856043] Lustre: Skipped 5 previous similar messages [ 1256.750078] LustreError: 90219:0:(obd_class.h:479:obd_check_dev()) Device 19 not setup [ 1256.753203] LustreError: 90219:0:(obd_class.h:479:obd_check_dev()) Skipped 9 previous similar messages [ 1256.801643] Lustre: server umount lustre-OST0000 complete [ 1264.273912] Lustre: server umount lustre-MDT0000 complete [ 1265.583356] LustreError: 87256:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1767386246 with bad export cookie 17080076598406147824 [ 1265.586404] LustreError: 87256:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) Skipped 1 previous similar message [ 1265.587087] LustreError: MGC192.168.203.108@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 1265.697145] Lustre: server umount lustre-MDT0001 complete [ 1269.136333] Lustre: DEBUG MARKER: == conf-sanity test 84: check recovery_hard_time ========= 15:37:28 (1767386248) [ 1271.843972] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1272.053924] LustreError: 91292:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 1273.672698] Lustre: DEBUG MARKER: oleg308-server.virtnet: executing set_default_debug -1 all [ 1277.409899] LustreError: 91292:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 1277.641130] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 1279.306348] Lustre: DEBUG MARKER: oleg308-server.virtnet: executing set_default_debug -1 all [ 1281.679089] Lustre: DEBUG MARKER: oleg308-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 1283.056436] Lustre: DEBUG MARKER: oleg308-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 1285.625087] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 1288.258838] Lustre: DEBUG MARKER: oleg308-server.virtnet: executing set_default_debug -1 all [ 1290.819917] Lustre: DEBUG MARKER: oleg308-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 1292.687515] Lustre: DEBUG MARKER: oleg308-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 [ 1297.956790] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 5 sec [ 1299.748615] Lustre: DEBUG MARKER: oleg308-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid 50 [ 1299.867381] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [ 1303.120341] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 1303.161825] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 1303.249328] Lustre: lustre-OST0001: new disk, initializing [ 1303.252412] Lustre: Skipped 2 previous similar messages [ 1303.255875] Lustre: srv-lustre-OST0001: No data found on store. Initialize space. [ 1303.259806] Lustre: Skipped 3 previous similar messages [ 1305.990127] Lustre: DEBUG MARKER: oleg308-server.virtnet: executing set_default_debug -1 all [ 1309.581787] Lustre: DEBUG MARKER: oleg308-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0001-osc-[-0-9a-f]*.ost_server_uuid [ 1311.347942] Lustre: DEBUG MARKER: oleg308-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0001-osc-MDT0000.ost_server_uuid 50 [ 1313.283250] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x00000002c0000400-0x0000000300000400]:1:ost [ 1313.288470] Lustre: Skipped 2 previous similar messages [ 1313.291064] Lustre: cli-lustre-OST0001-super: Allocated super-sequence [0x00000002c0000400-0x0000000300000400]:1:ost] [ 1313.318766] Lustre: lustre-OST0001-osc-MDT0000: update sequence from 0x100010000 to 0x2c0000401 [ 1313.503688] Lustre: DEBUG MARKER: os[cp].lustre-OST0001-osc-MDT0000.ost_server_uuid in FULL state after 2 sec [ 1314.975888] Lustre: DEBUG MARKER: oleg308-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0001-osc-MDT0001.ost_server_uuid 50 [ 1315.060615] Lustre: DEBUG MARKER: os[cp].lustre-OST0001-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [ 1325.021817] Lustre: DEBUG MARKER: mds1 REPLAY BARRIER on lustre-MDT0000 [ 1331.373994] Lustre: Failing over lustre-MDT0000 [ 1331.714754] LustreError: 96229:0:(obd_class.h:479:obd_check_dev()) Device 28 not setup [ 1331.716786] LustreError: 96229:0:(obd_class.h:479:obd_check_dev()) Skipped 17 previous similar messages [ 1331.749724] Lustre: server umount lustre-MDT0000 complete [ 1333.733054] Lustre: lustre-MDT0000-lwp-OST0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 1333.735197] LustreError: 91291:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 1333.741152] Lustre: Skipped 6 previous similar messages [ 1333.749334] LustreError: 91291:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 2 previous similar messages [ 1348.052162] LDISKFS-fs (dm-0): 4 truncates cleaned up [ 1348.054601] LDISKFS-fs (dm-0): recovery complete [ 1348.059509] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1348.115919] LustreError: MGC192.168.203.108@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 1348.277803] Lustre: lustre-MDT0000: in recovery but waiting for the first client to connect [ 1349.869198] Lustre: DEBUG MARKER: oleg308-server.virtnet: executing set_default_debug -1 all [ 1352.116767] Lustre: lustre-MDT0000: Will be in recovery for at least 1:00, or until 3 clients reconnect [ 1352.117964] Lustre: 91288:0:(ldlm_lib.c:2068:extend_recovery_timer()) lustre-MDT0000: extended recovery timer reached hard limit: 60, extend: 0 [ 1353.699474] Lustre: 91292:0:(ldlm_lib.c:2068:extend_recovery_timer()) lustre-MDT0000: extended recovery timer reached hard limit: 60, extend: 0 [ 1353.706579] Lustre: 91292:0:(ldlm_lib.c:2068:extend_recovery_timer()) Skipped 1 previous similar message [ 1353.722079] Lustre: lustre-MDT0000-lwp-OST0000: Connection restored to 0@lo (at 0@lo) [ 1353.750791] LustreError: 96980:0:(ldlm_lib.c:2681:replay_request_or_update()) cfs_fail_timeout id 709 sleeping for 300ms [ 1354.071139] LustreError: 96980:0:(ldlm_lib.c:2681:replay_request_or_update()) cfs_fail_timeout id 709 awake [ 1354.393598] LustreError: 96980:0:(ldlm_lib.c:2681:replay_request_or_update()) cfs_fail_timeout id 709 sleeping for 300ms [ 1354.397348] LustreError: 96980:0:(ldlm_lib.c:2681:replay_request_or_update()) Skipped 1 previous similar message [ 1354.711093] LustreError: 96980:0:(ldlm_lib.c:2681:replay_request_or_update()) cfs_fail_timeout id 709 awake [ 1354.714546] LustreError: 96980:0:(ldlm_lib.c:2681:replay_request_or_update()) Skipped 1 previous similar message [ 1354.718581] Lustre: 96980:0:(ldlm_lib.c:2068:extend_recovery_timer()) lustre-MDT0000: extended recovery timer reached hard limit: 60, extend: 1 [ 1354.723942] Lustre: 96980:0:(ldlm_lib.c:2068:extend_recovery_timer()) Skipped 7 previous similar messages [ 1355.666372] LustreError: 96980:0:(ldlm_lib.c:2681:replay_request_or_update()) cfs_fail_timeout id 709 sleeping for 300ms [ 1355.675451] LustreError: 96980:0:(ldlm_lib.c:2681:replay_request_or_update()) Skipped 3 previous similar messages [ 1355.999137] LustreError: 96980:0:(ldlm_lib.c:2681:replay_request_or_update()) cfs_fail_timeout id 709 awake [ 1356.005697] LustreError: 96980:0:(ldlm_lib.c:2681:replay_request_or_update()) Skipped 3 previous similar messages [ 1356.962238] Lustre: 96980:0:(ldlm_lib.c:2068:extend_recovery_timer()) lustre-MDT0000: extended recovery timer reached hard limit: 60, extend: 1 [ 1356.968648] Lustre: 96980:0:(ldlm_lib.c:2068:extend_recovery_timer()) Skipped 6 previous similar messages [ 1357.911475] LustreError: 96980:0:(ldlm_lib.c:2681:replay_request_or_update()) cfs_fail_timeout id 709 sleeping for 300ms [ 1357.913699] LustreError: 96980:0:(ldlm_lib.c:2681:replay_request_or_update()) Skipped 6 previous similar messages [ 1358.231065] LustreError: 96980:0:(ldlm_lib.c:2681:replay_request_or_update()) cfs_fail_timeout id 709 awake [ 1358.232849] LustreError: 96980:0:(ldlm_lib.c:2681:replay_request_or_update()) Skipped 6 previous similar messages [ 1361.048165] Lustre: 96980:0:(ldlm_lib.c:2068:extend_recovery_timer()) lustre-MDT0000: extended recovery timer reached hard limit: 60, extend: 1 [ 1361.055551] Lustre: 96980:0:(ldlm_lib.c:2068:extend_recovery_timer()) Skipped 12 previous similar messages [ 1362.001855] LustreError: 96980:0:(ldlm_lib.c:2681:replay_request_or_update()) cfs_fail_timeout id 709 sleeping for 300ms [ 1362.007587] LustreError: 96980:0:(ldlm_lib.c:2681:replay_request_or_update()) Skipped 12 previous similar messages [ 1362.327113] LustreError: 96980:0:(ldlm_lib.c:2681:replay_request_or_update()) cfs_fail_timeout id 709 awake [ 1362.331759] LustreError: 96980:0:(ldlm_lib.c:2681:replay_request_or_update()) Skipped 12 previous similar messages [ 1369.200714] Lustre: 96980:0:(ldlm_lib.c:2068:extend_recovery_timer()) lustre-MDT0000: extended recovery timer reached hard limit: 60, extend: 1 [ 1369.209837] Lustre: 96980:0:(ldlm_lib.c:2068:extend_recovery_timer()) Skipped 25 previous similar messages [ 1370.153800] LustreError: 96980:0:(ldlm_lib.c:2681:replay_request_or_update()) cfs_fail_timeout id 709 sleeping for 300ms [ 1370.158847] LustreError: 96980:0:(ldlm_lib.c:2681:replay_request_or_update()) Skipped 25 previous similar messages [ 1370.471137] LustreError: 96980:0:(ldlm_lib.c:2681:replay_request_or_update()) cfs_fail_timeout id 709 awake [ 1370.477355] LustreError: 96980:0:(ldlm_lib.c:2681:replay_request_or_update()) Skipped 25 previous similar messages [ 1385.504054] Lustre: 96980:0:(ldlm_lib.c:2068:extend_recovery_timer()) lustre-MDT0000: extended recovery timer reached hard limit: 60, extend: 1 [ 1385.510238] Lustre: 96980:0:(ldlm_lib.c:2068:extend_recovery_timer()) Skipped 51 previous similar messages [ 1386.450442] LustreError: 96980:0:(ldlm_lib.c:2681:replay_request_or_update()) cfs_fail_timeout id 709 sleeping for 300ms [ 1386.456671] LustreError: 96980:0:(ldlm_lib.c:2681:replay_request_or_update()) Skipped 51 previous similar messages [ 1386.775135] LustreError: 96980:0:(ldlm_lib.c:2681:replay_request_or_update()) cfs_fail_timeout id 709 awake [ 1386.779689] LustreError: 96980:0:(ldlm_lib.c:2681:replay_request_or_update()) Skipped 51 previous similar messages [ 1412.063131] Lustre: 96980:0:(ldlm_lib.c:2377:target_recovery_overseer()) lustre-MDT0000 recovery is aborted by hard timeout [ 1412.068445] Lustre: 96980:0:(ldlm_lib.c:2387:target_recovery_overseer()) recovery is aborted, evict exports in recovery [ 1412.073529] Lustre: 96980:0:(genops.c:1618:class_disconnect_stale_exports()) lustre-MDT0000: disconnect stale client 15138340-fc74-4103-a6bd-e7e507c6859d@192.168.203.8@tcp [ 1412.080692] Lustre: lustre-MDT0000: disconnecting 1 stale clients [ 1412.084656] Lustre: 96980:0:(ldlm_lib.c:1897:abort_req_replay_queue()) @@@ aborted: req@00000000a6d6a27e x1853238420770688/t0(8589934783) o101->15138340-fc74-4103-a6bd-e7e507c6859d@192.168.203.8@tcp:108/0 lens 592/0 e 0 to 0 dl 1767386403 ref 1 fl Complete:/604/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:0 [ 1412.100359] Lustre: lustre-MDT0000-osd: cancel update llog [0x200000400:0x1:0x0] [ 1412.106211] Lustre: lustre-MDT0000: Denying connection for new client 15138340-fc74-4103-a6bd-e7e507c6859d (at 192.168.203.8@tcp), waiting for 3 known clients (2 recovered, 0 in progress, and 1 evicted) to recover in 0:00 [ 1412.112574] Lustre: lustre-MDT0001-osp-MDT0000: cancel update llog [0x240000401:0x1:0x0] [ 1412.138780] Lustre: lustre-MDT0000-osp-MDT0001: Connection restored to 0@lo (at 0@lo) [ 1412.143963] Lustre: Skipped 2 previous similar messages [ 1412.148178] Lustre: lustre-MDT0000: Recovery over after 1:00, of 3 clients 2 recovered and 1 was evicted. [ 1412.173347] Lustre: lustre-OST0001: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x2c0000401:49 to 0x2c0000401:545) [ 1412.173423] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:80 to 0x280000401:577) [ 1420.262892] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 1420.267004] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 1420.269133] Lustre: Skipped 1 previous similar message [ 1420.272056] Lustre: Skipped 5 previous similar messages [ 1425.421338] Lustre: server umount lustre-OST0000 complete [ 1427.935799] LustreError: lustre-OST0001-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 1427.938337] LustreError: Skipped 1 previous similar message [ 1430.501346] LustreError: 93163:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-OST0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 1430.510904] LustreError: 93163:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 15 previous similar messages [ 1441.247072] Lustre: lustre-OST0001 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 1441.301542] Lustre: server umount lustre-OST0001 complete [ 1457.119124] Lustre: lustre-MDT0000 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 1457.219699] Lustre: server umount lustre-MDT0000 complete [ 1458.894769] LustreError: 96230:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1767386439 with bad export cookie 17080076598406198994 [ 1458.901047] LustreError: 96230:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 1458.901125] LustreError: MGC192.168.203.108@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 1459.041436] Lustre: server umount lustre-MDT0001 complete [ 1462.889937] Lustre: DEBUG MARKER: == conf-sanity test 85: osd_ost init: fail ea_fid_set ==== 15:40:42 (1767386442) [ 1466.467488] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 1466.475060] Lustre: *** cfs_fail_loc=197, val=0*** [ 1481.951744] LustreError: Failed to get MGS log lustre-sptlrpc and no local copy. [ 1487.071350] LustreError: 98942:0:(mgc_request.c:233:do_config_log_add()) MGC192.168.203.108@tcp: failed processing log, type 4: rc = -110 [ 1492.201173] Lustre: *** cfs_fail_loc=197, val=0*** [ 1492.203908] Lustre: Skipped 65 previous similar messages [ 1512.671607] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 1512.675722] Lustre: Skipped 7 previous similar messages [ 1514.559674] Lustre: DEBUG MARKER: oleg308-server.virtnet: executing set_default_debug -1 all [ 1516.911223] Lustre: DEBUG MARKER: oleg308-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 1517.894714] LustreError: 99605:0:(obd_class.h:479:obd_check_dev()) Device 1 not setup [ 1517.902724] LustreError: 99605:0:(obd_class.h:479:obd_check_dev()) Skipped 33 previous similar messages [ 1517.961320] Lustre: server umount lustre-OST0000 complete [ 1521.223863] Lustre: DEBUG MARKER: == conf-sanity test 86: Replacing mkfs.lustre -G option == 15:41:40 (1767386500) [ 1535.614842] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1535.799605] LustreError: 100448:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 1535.805482] LustreError: 100448:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 4 previous similar messages [ 1537.439753] Lustre: DEBUG MARKER: oleg308-server.virtnet: executing set_default_debug -1 all [ 1541.086207] Lustre: DEBUG MARKER: oleg308-client.virtnet: executing wait_import_state_mount (FULL|IDLE) mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 1543.468545] LustreError: 100479:0:(osp_object.c:617:osp_attr_get()) lustre-MDT0001-osp-MDT0000: osp_attr_get update error [0x200000009:0x1:0x0]: rc = -5 [ 1543.475288] LustreError: 100479:0:(lod_sub_object.c:917:lod_sub_prep_llog()) lustre-MDT0000-mdtlov: can't get id from catalogs: rc = -5 [ 1543.481074] LustreError: 100479:0:(lod_dev.c:510:lod_sub_recovery_thread()) lustre-MDT0001-osp-MDT0000: get update log duration 8, retries 0, failed: rc = -5 [ 1543.622094] Lustre: server umount lustre-MDT0000 complete [ 1550.566072] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 1557.655486] Lustre: DEBUG MARKER: oleg308-server.virtnet: executing set_hostid [ 1562.050427] Lustre: DEBUG MARKER: oleg308-server.virtnet: executing load_modules_local [ 1566.173201] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 1569.179631] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 1572.171311] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 1574.686929] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 1577.503584] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 1577.545908] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1577.666853] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 1577.713474] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 1577.777069] Lustre: lustre-MDT0000: new disk, initializing [ 1577.824136] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 1579.243878] Lustre: DEBUG MARKER: oleg308-server.virtnet: executing set_default_debug -1 all [ 1584.715064] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 1584.758360] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 1584.812733] Lustre: 104511:0:(mgs_llog.c:1348:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 1584.956641] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 1586.687582] Lustre: DEBUG MARKER: oleg308-server.virtnet: executing set_default_debug -1 all [ 1590.562904] Lustre: DEBUG MARKER: oleg308-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 1592.227243] Lustre: DEBUG MARKER: oleg308-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 1595.318056] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 1595.353493] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 1597.815362] Lustre: DEBUG MARKER: oleg308-server.virtnet: executing set_default_debug -1 all [ 1601.703683] Lustre: DEBUG MARKER: oleg308-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 1603.512949] Lustre: DEBUG MARKER: oleg308-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 [ 1605.189308] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 1605.695192] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 2 sec [ 1607.349119] Lustre: DEBUG MARKER: oleg308-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid 50 [ 1607.417378] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [ 1610.214786] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 1610.215401] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 1610.222537] Lustre: Skipped 4 previous similar messages [ 1610.227694] LustreError: Skipped 1 previous similar message [ 1614.244687] Lustre: server umount lustre-OST0000 complete [ 1622.256818] Lustre: server umount lustre-MDT0000 complete [ 1623.866119] LustreError: 104503:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1767386604 with bad export cookie 17080076598406209704 [ 1623.871471] LustreError: MGC192.168.203.108@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 1623.875667] LustreError: 104503:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 1624.005841] Lustre: server umount lustre-MDT0001 complete [ 1626.777986] Lustre: DEBUG MARKER: == conf-sanity test 87: check if MDT inode can hold EAs with N stripes properly ========================================================== 15:43:26 (1767386606) [ 1630.071098] Lustre: DEBUG MARKER: oleg308-server.virtnet: executing unload_modules_local [ 1631.166347] Key type lgssc unregistered [ 1631.330615] LNet: 108598:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 1631.335433] LNetError: 108598:0:(acceptor.c:264:lnet_acceptor_remove_socket()) Interface ens2 not found [ 1631.343608] LNet: Removed LNI 192.168.203.108@tcp [ 1631.801797] Key type .llcrypt unregistered [ 1631.804161] Key type ._llcrypt unregistered [ 1639.455478] Key type ._llcrypt registered [ 1639.456525] Key type .llcrypt registered [ 1639.493972] Lustre: DEBUG MARKER: oleg308-server.virtnet: executing set_hostid [ 1647.382497] Lustre: DEBUG MARKER: oleg308-server.virtnet: executing load_modules_local [ 1647.837429] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 1647.847833] alg: No test for adler32 (adler32-zlib) [ 1648.747670] Lustre: Lustre: Build Version: 2.17.0_RC4_2_g45eb11f [ 1648.875966] LNet: Added LNI 192.168.203.108@tcp [8/256/0/180] [ 1650.471287] Key type lgssc registered [ 1651.042053] Lustre: Echo OBD driver; http://www.lustre.org/ [ 1655.355660] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 1658.630166] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 1661.411646] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 1664.164436] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 1666.502406] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 1668.853560] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 1668.881352] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 1668.889435] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1669.980621] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 1669.990642] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 1670.024793] Lustre: lustre-MDT0000: new disk, initializing [ 1670.046790] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 1670.053057] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 1671.745909] Lustre: DEBUG MARKER: oleg308-server.virtnet: executing set_default_debug -1 all [ 1676.791324] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 1676.828811] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 1676.966538] Lustre: lustre-OST0000: new disk, initializing [ 1676.971751] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 1676.978188] Lustre: Skipped 1 previous similar message [ 1677.018470] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 1679.552558] Lustre: DEBUG MARKER: oleg308-server.virtnet: executing set_default_debug -1 all [ 1683.987614] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:0:ost [ 1683.993880] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:0:ost] [ 1684.011068] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x240000400 [ 1684.240669] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 1684.277665] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 1684.354969] Lustre: lustre-OST0001: new disk, initializing [ 1684.357691] Lustre: srv-lustre-OST0001: No data found on store. Initialize space. [ 1684.393646] Lustre: lustre-OST0001: Imperative Recovery not enabled, recovery window 60-180 [ 1686.665303] Lustre: DEBUG MARKER: oleg308-server.virtnet: executing set_default_debug -1 all [ 1690.651798] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:1:ost [ 1690.655517] Lustre: cli-lustre-OST0001-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:1:ost] [ 1690.666766] Lustre: lustre-OST0001-osc-MDT0000: update sequence from 0x100010000 to 0x280000400 [ 1691.164955] LustreError: 114151:0:(obd_class.h:479:obd_check_dev()) Device 17 not setup [ 1691.239541] Lustre: server umount lustre-MDT0000 complete [ 1693.673340] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: (null) [ 1695.867192] LustreError: 111909:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1767386676 with bad export cookie 5542823995042478106 [ 1695.877762] LustreError: MGC192.168.203.108@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 1705.465190] LustreError: 114746:0:(obd_class.h:479:obd_check_dev()) Device 12 not setup [ 1705.470551] LustreError: 114746:0:(obd_class.h:479:obd_check_dev()) Skipped 5 previous similar messages [ 1705.491578] Lustre: server umount lustre-OST0000 complete [ 1716.343654] LustreError: 114947:0:(obd_class.h:479:obd_check_dev()) Device 2 not setup [ 1716.349039] LustreError: 114947:0:(obd_class.h:479:obd_check_dev()) Skipped 1 previous similar message [ 1716.415163] Lustre: server umount lustre-OST0001 complete [ 1719.658908] Lustre: DEBUG MARKER: == conf-sanity test 88: check the default mount options can be overridden ========================================================== 15:44:59 (1767386699) [ 1723.443516] Lustre: DEBUG MARKER: oleg308-server.virtnet: executing load_modules_local [ 1726.631471] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 1726.969140] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 1728.956545] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 1729.407152] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 1732.040061] Lustre: DEBUG MARKER: == conf-sanity test 89: check tunefs --param and --erase-param{s} options ========================================================== 15:45:11 (1767386711) [ 1736.983224] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 1737.612309] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 1738.567944] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 1739.313494] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 1739.619525] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 1740.246496] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 1747.100052] Lustre: DEBUG MARKER: oleg308-server.virtnet: executing set_hostid [ 1750.579122] Lustre: DEBUG MARKER: oleg308-server.virtnet: executing load_modules_local [ 1754.318609] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 1757.467821] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 1759.867935] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 1762.029250] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 1764.913450] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 1764.946983] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1765.052770] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 1765.072973] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 1765.121438] Lustre: lustre-MDT0000: new disk, initializing [ 1765.145267] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 1765.151782] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 1766.755813] Lustre: DEBUG MARKER: oleg308-server.virtnet: executing set_default_debug -1 all [ 1771.126232] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 1771.149305] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 1771.180333] Lustre: 120262:0:(mgs_llog.c:1348:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 1771.200318] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [ 1771.202822] Lustre: Skipped 1 previous similar message [ 1771.238055] Lustre: lustre-MDT0001: new disk, initializing [ 1771.267133] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 1771.277514] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [ 1771.281076] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 1772.436632] Lustre: DEBUG MARKER: oleg308-server.virtnet: executing set_default_debug -1 all [ 1776.198264] Lustre: DEBUG MARKER: oleg308-client.virtnet: executing wait_import_state_mount FULL failover.node=192.0.2.254@tcp0,mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 1778.103199] Lustre: DEBUG MARKER: oleg308-client.virtnet: executing wait_import_state_mount FULL failover.node=192.0.2.254@tcp0,mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 1781.312497] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 1781.342312] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 1781.450385] Lustre: lustre-OST0000: new disk, initializing [ 1781.453141] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 1781.481707] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 1781.749391] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [ 1781.755498] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 1781.812822] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 1783.880586] Lustre: DEBUG MARKER: oleg308-server.virtnet: executing set_default_debug -1 all [ 1787.795435] Lustre: DEBUG MARKER: oleg308-client.virtnet: executing wait_import_state_mount (FULL|IDLE) failover.node=192.0.2.254@tcp0,osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 1789.175486] Lustre: DEBUG MARKER: oleg308-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 [ 1789.251259] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [ 1790.737467] Lustre: DEBUG MARKER: oleg308-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid 50 [ 1790.820575] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [ 1791.973092] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 1791.973793] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 1791.981751] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 1791.985583] LustreError: Skipped 1 previous similar message [ 1797.090092] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 1797.092430] Lustre: Skipped 2 previous similar messages [ 1797.743102] LustreError: 123200:0:(obd_class.h:479:obd_check_dev()) Device 19 not setup [ 1797.745466] LustreError: 123200:0:(obd_class.h:479:obd_check_dev()) Skipped 3 previous similar messages [ 1797.782779] Lustre: server umount lustre-OST0000 complete [ 1802.208878] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 1802.209012] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 1802.209718] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 1802.211770] Lustre: Skipped 2 previous similar messages [ 1805.440743] LustreError: 123401:0:(obd_class.h:479:obd_check_dev()) Device 22 not setup [ 1805.442562] LustreError: 123401:0:(obd_class.h:479:obd_check_dev()) Skipped 3 previous similar messages [ 1805.467640] Lustre: server umount lustre-MDT0000 complete [ 1806.567047] LustreError: 120254:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1767386787 with bad export cookie 5542823995042479331 [ 1806.570080] LustreError: MGC192.168.203.108@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 1806.571748] LustreError: 120254:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 1806.683849] Lustre: server umount lustre-MDT0001 complete [ 1809.165625] Lustre: DEBUG MARKER: == conf-sanity test 90a: check max_mod_rpcs_in_flight is enforced ========================================================== 15:46:28 (1767386788) [ 1811.890142] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1812.137130] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 1813.956315] Lustre: DEBUG MARKER: oleg308-server.virtnet: executing set_default_debug -1 all [ 1817.569413] LustreError: 124273:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 1817.580386] LustreError: 124273:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 2 previous similar messages [ 1817.703225] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 1819.465260] Lustre: DEBUG MARKER: oleg308-server.virtnet: executing set_default_debug -1 all [ 1821.501154] Lustre: DEBUG MARKER: oleg308-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 1823.209702] Lustre: DEBUG MARKER: oleg308-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 1826.029302] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 1828.617600] Lustre: DEBUG MARKER: oleg308-server.virtnet: executing set_default_debug -1 all [ 1831.069943] Lustre: DEBUG MARKER: oleg308-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 1838.826459] Lustre: *** cfs_fail_loc=159, val=0*** [ 1854.426724] Lustre: lustre-MDT0000: Client 40c210e9-8b7d-4b1c-af01-152b14de92e6 (at 192.168.203.8@tcp) reconnecting [ 1855.178071] Lustre: *** cfs_fail_loc=159, val=0*** [ 1855.181099] Lustre: Skipped 5 previous similar messages [ 1871.824746] Lustre: lustre-MDT0000: Client 40c210e9-8b7d-4b1c-af01-152b14de92e6 (at 192.168.203.8@tcp) reconnecting [ 1874.401358] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 1874.406023] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 1874.414085] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 1874.417703] Lustre: Skipped 1 previous similar message [ 1877.475518] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 1877.475682] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 1879.473161] LustreError: 126760:0:(obd_class.h:479:obd_check_dev()) Device 21 not setup [ 1879.476204] LustreError: 126760:0:(obd_class.h:479:obd_check_dev()) Skipped 13 previous similar messages [ 1879.523462] Lustre: server umount lustre-OST0000 complete [ 1882.594057] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 1882.595571] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 1882.601436] Lustre: Skipped 1 previous similar message [ 1882.607595] Lustre: Skipped 3 previous similar messages [ 1887.473448] Lustre: server umount lustre-MDT0000 complete [ 1887.714071] LustreError: 124273:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 1887.723633] LustreError: 124273:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 1 previous similar message [ 1889.255228] LustreError: 124992:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1767386869 with bad export cookie 5542823995042480199 [ 1889.262252] LustreError: MGC192.168.203.108@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 1889.263114] LustreError: 124992:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 1889.403828] Lustre: server umount lustre-MDT0001 complete [ 1893.923582] Lustre: DEBUG MARKER: oleg308-server.virtnet: executing unload_modules_local [ 1895.058694] Key type lgssc unregistered [ 1895.208432] LNet: 127695:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 1895.211051] LNetError: 127695:0:(acceptor.c:264:lnet_acceptor_remove_socket()) Interface ens2 not found [ 1895.218679] LNet: Removed LNI 192.168.203.108@tcp [ 1895.577509] Key type .llcrypt unregistered [ 1895.578835] Key type ._llcrypt unregistered [ 1906.373530] Key type ._llcrypt registered [ 1906.374682] Key type .llcrypt registered [ 1906.414870] Lustre: DEBUG MARKER: oleg308-server.virtnet: executing load_modules_local [ 1906.711297] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 1906.717798] alg: No test for adler32 (adler32-zlib) [ 1907.575807] Lustre: Lustre: Build Version: 2.17.0_RC4_2_g45eb11f [ 1907.657702] LNet: Added LNI 192.168.203.108@tcp [8/256/0/180] [ 1909.239080] Key type lgssc registered [ 1909.548066] Lustre: Echo OBD driver; http://www.lustre.org/ [ 1912.382499] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 1912.391381] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1913.557557] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 1915.077881] Lustre: DEBUG MARKER: oleg308-server.virtnet: executing set_default_debug -1 all [ 1918.456382] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 1918.616865] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 1920.141331] Lustre: DEBUG MARKER: oleg308-server.virtnet: executing set_default_debug -1 all [ 1922.180545] Lustre: DEBUG MARKER: oleg308-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 1923.572141] Lustre: DEBUG MARKER: oleg308-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 1926.541932] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 1926.718828] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 1929.143280] Lustre: DEBUG MARKER: oleg308-server.virtnet: executing set_default_debug -1 all [ 1931.616136] Lustre: DEBUG MARKER: oleg308-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 1939.052646] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:42 to 0x280000401:65) [ 1947.109870] Lustre: *** cfs_fail_loc=159, val=0*** [ 1963.472052] Lustre: lustre-MDT0000: Client 74868816-246c-49f1-bc26-f74762da040d (at 192.168.203.8@tcp) reconnecting [ 1964.467465] Lustre: *** cfs_fail_loc=159, val=0*** [ 1980.878143] Lustre: lustre-MDT0001: Client 74868816-246c-49f1-bc26-f74762da040d (at 192.168.203.8@tcp) reconnecting [ 1981.527451] Lustre: *** cfs_fail_loc=159, val=0*** [ 1981.529291] Lustre: Skipped 3 previous similar messages [ 1997.776093] Lustre: lustre-MDT0001: Client 74868816-246c-49f1-bc26-f74762da040d (at 192.168.203.8@tcp) reconnecting [ 1999.331321] Lustre: 131793:0:(mdt_handler.c:64:mdt_max_mod_rpcs_per_client_set()) max_mod_rpcs_per_client is deprecated, set mdt.*.max_mod_rpcs_in_flight parameter instead [ 2000.267464] Lustre: *** cfs_fail_loc=159, val=0*** [ 2000.269674] Lustre: Skipped 3 previous similar messages [ 2016.719884] Lustre: lustre-MDT0000: Client ffa7bebb-8f50-41a6-81fe-f33863d8f708 (at 192.168.203.8@tcp) reconnecting [ 2017.522956] Lustre: *** cfs_fail_loc=159, val=0*** [ 2017.525708] Lustre: Skipped 13 previous similar messages [ 2033.089730] Lustre: lustre-MDT0000: Client ffa7bebb-8f50-41a6-81fe-f33863d8f708 (at 192.168.203.8@tcp) reconnecting [ 2033.633483] Lustre: 132083:0:(mdt_handler.c:64:mdt_max_mod_rpcs_per_client_set()) max_mod_rpcs_per_client is deprecated, set mdt.*.max_mod_rpcs_in_flight parameter instead [ 2036.710095] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 2036.710941] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2036.714763] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 2036.728085] Lustre: Skipped 1 previous similar message [ 2041.140170] LustreError: 132181:0:(obd_class.h:479:obd_check_dev()) Device 21 not setup [ 2041.197536] Lustre: server umount lustre-OST0000 complete [ 2046.946600] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2046.946741] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 2046.947814] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 2046.947823] Lustre: Skipped 1 previous similar message [ 2046.955910] Lustre: Skipped 1 previous similar message [ 2046.960602] LustreError: Skipped 1 previous similar message [ 2048.826918] LustreError: 132382:0:(obd_class.h:479:obd_check_dev()) Device 9 not setup [ 2048.828435] LustreError: 132382:0:(obd_class.h:479:obd_check_dev()) Skipped 3 previous similar messages [ 2048.853836] Lustre: server umount lustre-MDT0000 complete [ 2050.369265] LustreError: 129048:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1767387031 with bad export cookie 8967313740415616383 [ 2050.374189] LustreError: MGC192.168.203.108@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 2050.378877] LustreError: 129048:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 2050.478950] LustreError: 132583:0:(obd_class.h:479:obd_check_dev()) Device 16 not setup [ 2050.481094] LustreError: 132583:0:(obd_class.h:479:obd_check_dev()) Skipped 5 previous similar messages [ 2050.538471] Lustre: server umount lustre-MDT0001 complete [ 2054.141415] Lustre: DEBUG MARKER: oleg308-server.virtnet: executing unload_modules_local [ 2055.487691] Key type lgssc unregistered [ 2055.630612] LNet: 133114:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 2055.635438] LNetError: 133114:0:(acceptor.c:264:lnet_acceptor_remove_socket()) Interface ens2 not found [ 2055.653576] LNet: Removed LNI 192.168.203.108@tcp [ 2056.107102] Key type .llcrypt unregistered [ 2056.109390] Key type ._llcrypt unregistered [ 2068.373719] Key type ._llcrypt registered [ 2068.374876] Key type .llcrypt registered [ 2068.423385] Lustre: DEBUG MARKER: oleg308-server.virtnet: executing load_modules_local [ 2068.719541] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 2068.753551] alg: No test for adler32 (adler32-zlib) [ 2069.615859] Lustre: Lustre: Build Version: 2.17.0_RC4_2_g45eb11f [ 2069.703944] LNet: Added LNI 192.168.203.108@tcp [8/256/0/180] [ 2071.287146] Key type lgssc registered [ 2071.804163] Lustre: Echo OBD driver; http://www.lustre.org/ [ 2075.399968] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 2075.412645] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 2076.639205] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 2078.358938] Lustre: DEBUG MARKER: oleg308-server.virtnet: executing set_default_debug -1 all [ 2082.433757] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 2082.616410] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 2084.192666] Lustre: DEBUG MARKER: oleg308-server.virtnet: executing set_default_debug -1 all [ 2086.410322] Lustre: DEBUG MARKER: oleg308-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 2088.246499] Lustre: DEBUG MARKER: oleg308-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 2091.531395] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 2091.696883] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 2092.720906] Lustre: lustre-OST0000: new connection from lustre-MDT0001-mdtlov (cleaning up unused objects from 0x280000400:72 to 0x280000400:97) [ 2094.023297] Lustre: DEBUG MARKER: oleg308-server.virtnet: executing set_default_debug -1 all [ 2095.990049] Lustre: DEBUG MARKER: oleg308-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 2101.934948] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:85 to 0x280000401:129) [ 2109.477735] Lustre: 136762:0:(mdt_handler.c:64:mdt_max_mod_rpcs_per_client_set()) max_mod_rpcs_per_client is deprecated, set mdt.*.max_mod_rpcs_in_flight parameter instead [ 2110.359170] Lustre: 136810:0:(mdt_handler.c:64:mdt_max_mod_rpcs_per_client_set()) max_mod_rpcs_per_client is deprecated, set mdt.*.max_mod_rpcs_in_flight parameter instead [ 2111.049368] Lustre: Modifying parameter lustre.mdc.lustre-MDT0000-mdc-*.max_rpcs_in_flight=8 in log params [ 2112.598458] Lustre: 136958:0:(mgs_llog.c:1348:mgs_modify_param()) MGS: modify lustre/mdc.lustre-MDT0000-mdc-*.max_rpcs_in_flight=8 (mode = 0) failed: rc = -17 [ 2113.043442] Lustre: Modifying parameter lustre.mdc.lustre-MDT0000-mdc-*.max_mod_rpcs_in_flight=7 in log params [ 2113.048733] Lustre: Skipped 1 previous similar message [ 2113.490559] Lustre: 137056:0:(mdt_handler.c:64:mdt_max_mod_rpcs_per_client_set()) max_mod_rpcs_per_client is deprecated, set mdt.*.max_mod_rpcs_in_flight parameter instead [ 2116.787113] LustreError: 137153:0:(obd_class.h:479:obd_check_dev()) Device 21 not setup [ 2116.845342] Lustre: server umount lustre-OST0000 complete [ 2118.627560] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 2118.634305] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2118.645486] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 2122.720615] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 2122.721156] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2127.841291] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 2127.845122] Lustre: Skipped 2 previous similar messages [ 2128.869061] LustreError: 137355:0:(obd_class.h:479:obd_check_dev()) Device 9 not setup [ 2128.874573] LustreError: 137355:0:(obd_class.h:479:obd_check_dev()) Skipped 3 previous similar messages [ 2128.914636] Lustre: server umount lustre-MDT0000 complete [ 2130.721047] LustreError: 134467:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1767387111 with bad export cookie 2806385632565026924 [ 2130.727450] LustreError: MGC192.168.203.108@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 2130.731249] LustreError: 134467:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 2141.158072] LustreError: 137561:0:(obd_class.h:479:obd_check_dev()) Device 16 not setup [ 2141.163807] LustreError: 137561:0:(obd_class.h:479:obd_check_dev()) Skipped 5 previous similar messages [ 2141.238572] Lustre: server umount lustre-MDT0001 complete [ 2146.134757] Lustre: DEBUG MARKER: oleg308-server.virtnet: executing unload_modules_local [ 2147.585738] Key type lgssc unregistered [ 2147.754686] LNet: 138094:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 2147.759490] LNetError: 138094:0:(acceptor.c:264:lnet_acceptor_remove_socket()) Interface ens2 not found [ 2147.770570] LNet: Removed LNI 192.168.203.108@tcp [ 2148.240748] Key type .llcrypt unregistered [ 2148.243913] Key type ._llcrypt unregistered [ 2159.169330] Key type ._llcrypt registered [ 2159.170863] Key type .llcrypt registered [ 2159.226371] Lustre: DEBUG MARKER: oleg308-server.virtnet: executing load_modules_local [ 2159.592641] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 2159.607884] alg: No test for adler32 (adler32-zlib) [ 2160.461761] Lustre: Lustre: Build Version: 2.17.0_RC4_2_g45eb11f [ 2160.550323] LNet: Added LNI 192.168.203.108@tcp [8/256/0/180] [ 2162.143154] Key type lgssc registered [ 2162.630370] Lustre: Echo OBD driver; http://www.lustre.org/ [ 2165.676729] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 2165.688405] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 2166.909788] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 2168.570848] Lustre: DEBUG MARKER: oleg308-server.virtnet: executing set_default_debug -1 all [ 2172.245984] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 2172.377485] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 2173.838489] Lustre: DEBUG MARKER: oleg308-server.virtnet: executing set_default_debug -1 all [ 2176.421750] Lustre: DEBUG MARKER: oleg308-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 2178.128538] Lustre: DEBUG MARKER: oleg308-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 2180.945698] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 2181.132361] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 2183.209957] Lustre: lustre-OST0000: new connection from lustre-MDT0001-mdtlov (cleaning up unused objects from 0x280000400:72 to 0x280000400:129) [ 2183.460848] Lustre: DEBUG MARKER: oleg308-server.virtnet: executing set_default_debug -1 all [ 2185.651577] Lustre: DEBUG MARKER: oleg308-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 2191.404840] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:85 to 0x280000401:161) [ 2198.980118] Lustre: *** cfs_fail_loc=159, val=0*** [ 2214.347656] Lustre: lustre-MDT0000: Client 8d7b8605-0d53-4e27-b725-37b14d54477e (at 192.168.203.8@tcp) reconnecting [ 2216.934220] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 2216.934391] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2216.941901] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 2216.948390] Lustre: Skipped 1 previous similar message [ 2216.954352] Lustre: Skipped 1 previous similar message [ 2222.052081] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 2222.056453] Lustre: Skipped 1 previous similar message [ 2222.074362] LustreError: 141862:0:(obd_class.h:479:obd_check_dev()) Device 21 not setup [ 2222.126943] Lustre: server umount lustre-OST0000 complete [ 2227.169869] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2227.171508] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 2227.178451] Lustre: Skipped 1 previous similar message [ 2229.967536] LustreError: 142064:0:(obd_class.h:479:obd_check_dev()) Device 9 not setup [ 2229.971521] LustreError: 142064:0:(obd_class.h:479:obd_check_dev()) Skipped 3 previous similar messages [ 2230.002580] Lustre: server umount lustre-MDT0000 complete [ 2231.725873] LustreError: 139447:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1767387212 with bad export cookie 6573410793986235160 [ 2231.732135] LustreError: MGC192.168.203.108@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 2231.738072] LustreError: 139447:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 2231.831660] LustreError: 142266:0:(obd_class.h:479:obd_check_dev()) Device 16 not setup [ 2231.834947] LustreError: 142266:0:(obd_class.h:479:obd_check_dev()) Skipped 5 previous similar messages [ 2231.892497] Lustre: server umount lustre-MDT0001 complete [ 2236.747952] Lustre: DEBUG MARKER: oleg308-server.virtnet: executing unload_modules_local [ 2238.244870] Key type lgssc unregistered [ 2238.408603] LNet: 142797:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 2238.413552] LNetError: 142797:0:(acceptor.c:264:lnet_acceptor_remove_socket()) Interface ens2 not found [ 2238.424742] LNet: Removed LNI 192.168.203.108@tcp [ 2238.901696] Key type .llcrypt unregistered [ 2238.903840] Key type ._llcrypt unregistered [ 2250.891574] Key type ._llcrypt registered [ 2250.893522] Key type .llcrypt registered [ 2250.942425] Lustre: DEBUG MARKER: oleg308-server.virtnet: executing load_modules_local [ 2251.382736] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 2251.397833] alg: No test for adler32 (adler32-zlib) [ 2252.264680] Lustre: Lustre: Build Version: 2.17.0_RC4_2_g45eb11f [ 2252.353170] LNet: Added LNI 192.168.203.108@tcp [8/256/0/180] [ 2253.935150] Key type lgssc registered [ 2254.352508] Lustre: Echo OBD driver; http://www.lustre.org/ [ 2257.803087] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 2257.813354] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 2259.035936] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 2260.711151] Lustre: DEBUG MARKER: oleg308-server.virtnet: executing set_default_debug -1 all [ 2263.974836] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 2264.088425] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 2265.275594] Lustre: DEBUG MARKER: oleg308-server.virtnet: executing set_default_debug -1 all [ 2267.642348] Lustre: DEBUG MARKER: oleg308-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 2269.383192] Lustre: DEBUG MARKER: oleg308-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 2272.350244] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 2272.484224] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 2273.514561] Lustre: lustre-OST0000: new connection from lustre-MDT0001-mdtlov (cleaning up unused objects from 0x280000400:72 to 0x280000400:161) [ 2274.962622] Lustre: DEBUG MARKER: oleg308-server.virtnet: executing set_default_debug -1 all [ 2277.586550] Lustre: DEBUG MARKER: oleg308-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 2284.778340] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:170 to 0x280000401:193) [ 2302.643728] Lustre: lustre-OST0000: evicting bb537513-3f9a-4b74-bfca-007ac4a8b483 (at 192.168.203.8@tcp) by administrative request [ 2302.650534] Lustre: Skipped 1 previous similar message [ 2330.080465] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 2330.085220] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2330.093357] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 2331.107239] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 2331.109525] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2331.110669] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 2336.227127] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 2336.230520] Lustre: Skipped 2 previous similar messages [ 2341.343085] Lustre: lustre-OST0000 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 2341.345533] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 2341.347794] Lustre: Skipped 1 previous similar message [ 2341.361911] LustreError: 146976:0:(obd_class.h:479:obd_check_dev()) Device 21 not setup [ 2341.393826] Lustre: server umount lustre-OST0000 complete [ 2345.952398] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 2345.957161] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2345.965070] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 2356.704208] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 2356.706649] Lustre: Skipped 5 previous similar messages [ 2357.215177] Lustre: lustre-MDT0000 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 2357.252749] LustreError: 147178:0:(obd_class.h:479:obd_check_dev()) Device 9 not setup [ 2357.254839] LustreError: 147178:0:(obd_class.h:479:obd_check_dev()) Skipped 3 previous similar messages [ 2357.286488] Lustre: server umount lustre-MDT0000 complete [ 2358.851923] LustreError: 144151:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1767387339 with bad export cookie 3707681498563327024 [ 2358.858242] LustreError: MGC192.168.203.108@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 2358.859333] LustreError: 144151:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 2358.948918] LustreError: 147382:0:(obd_class.h:479:obd_check_dev()) Device 16 not setup [ 2358.951677] LustreError: 147382:0:(obd_class.h:479:obd_check_dev()) Skipped 5 previous similar messages [ 2359.007139] Lustre: server umount lustre-MDT0001 complete [ 2363.068767] Lustre: DEBUG MARKER: oleg308-server.virtnet: executing unload_modules_local [ 2364.098238] Key type lgssc unregistered [ 2364.273590] LNet: 147913:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 2364.278269] LNetError: 147913:0:(acceptor.c:264:lnet_acceptor_remove_socket()) Interface ens2 not found [ 2364.291578] LNet: Removed LNI 192.168.203.108@tcp [ 2364.680165] Key type .llcrypt unregistered [ 2364.681928] Key type ._llcrypt unregistered [ 2398.831544] Key type ._llcrypt registered [ 2398.833032] Key type .llcrypt registered [ 2398.885755] Lustre: DEBUG MARKER: oleg308-server.virtnet: executing load_modules_local [ 2399.197659] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 2399.249993] alg: No test for adler32 (adler32-zlib) [ 2400.134625] Lustre: Lustre: Build Version: 2.17.0_RC4_2_g45eb11f [ 2400.224198] LNet: Added LNI 192.168.203.108@tcp [8/256/0/180] [ 2401.815226] Key type lgssc registered [ 2402.327341] Lustre: Echo OBD driver; http://www.lustre.org/ [ 2405.813563] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 2405.826433] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 2406.985813] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 2408.416988] Lustre: DEBUG MARKER: oleg308-server.virtnet: executing set_default_debug -1 all [ 2412.732528] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 2412.927959] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 2414.738215] Lustre: DEBUG MARKER: oleg308-server.virtnet: executing set_default_debug -1 all [ 2417.316629] Lustre: DEBUG MARKER: oleg308-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 2419.158361] Lustre: DEBUG MARKER: oleg308-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 2422.409568] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 2422.570731] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 2423.591758] Lustre: lustre-OST0000: new connection from lustre-MDT0001-mdtlov (cleaning up unused objects from 0x280000400:72 to 0x280000400:193) [ 2424.647732] Lustre: DEBUG MARKER: oleg308-server.virtnet: executing set_default_debug -1 all [ 2426.887248] Lustre: DEBUG MARKER: oleg308-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 2431.788077] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:195 to 0x280000401:225) [ 2437.092365] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 2437.093158] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2437.099153] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 2437.105306] Lustre: Skipped 1 previous similar message [ 2437.111414] Lustre: Skipped 1 previous similar message [ 2441.645957] LustreError: 152612:0:(obd_class.h:479:obd_check_dev()) Device 21 not setup [ 2441.676639] Lustre: server umount lustre-OST0000 complete [ 2443.744434] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 2443.751034] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2443.762108] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 2447.329224] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 2447.330284] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2449.275271] LustreError: 152814:0:(obd_class.h:479:obd_check_dev()) Device 9 not setup [ 2449.277877] LustreError: 152814:0:(obd_class.h:479:obd_check_dev()) Skipped 3 previous similar messages [ 2449.303595] Lustre: server umount lustre-MDT0000 complete [ 2450.552338] LustreError: 152816:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1767387431 with bad export cookie 6489172201874555564 [ 2450.555301] LustreError: 152816:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) Skipped 3 previous similar messages [ 2450.555771] LustreError: MGC192.168.203.108@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 2450.641862] LustreError: 153017:0:(obd_class.h:479:obd_check_dev()) Device 16 not setup [ 2450.643755] LustreError: 153017:0:(obd_class.h:479:obd_check_dev()) Skipped 5 previous similar messages [ 2450.694673] Lustre: server umount lustre-MDT0001 complete [ 2455.461756] Lustre: DEBUG MARKER: oleg308-server.virtnet: executing unload_modules_local [ 2456.471362] Key type lgssc unregistered [ 2456.588316] LNet: 153548:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 2456.590320] LNetError: 153548:0:(acceptor.c:264:lnet_acceptor_remove_socket()) Interface ens2 not found [ 2456.597265] LNet: Removed LNI 192.168.203.108@tcp [ 2456.844037] Key type .llcrypt unregistered [ 2456.844980] Key type ._llcrypt unregistered [ 2467.812854] Key type ._llcrypt registered [ 2467.813977] Key type .llcrypt registered [ 2467.858305] Lustre: DEBUG MARKER: oleg308-server.virtnet: executing load_modules_local [ 2468.143740] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 2468.149592] alg: No test for adler32 (adler32-zlib) [ 2468.993390] Lustre: Lustre: Build Version: 2.17.0_RC4_2_g45eb11f [ 2469.071054] LNet: Added LNI 192.168.203.108@tcp [8/256/0/180] [ 2470.655206] Key type lgssc registered [ 2471.016599] Lustre: Echo OBD driver; http://www.lustre.org/ [ 2474.014566] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 2477.733847] Lustre: DEBUG MARKER: == conf-sanity test 100: check lshowmount lists MGS, MDT, OST and 0@lo ========================================================== 15:57:36 (1767387456) [ 2484.270213] Lustre: DEBUG MARKER: oleg308-server.virtnet: executing set_hostid [ 2488.990156] Lustre: DEBUG MARKER: oleg308-server.virtnet: executing load_modules_local [ 2493.483382] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 2496.659711] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 2499.630909] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 2502.892710] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 2506.168197] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 2506.201809] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 2506.211143] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 2507.319254] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 2507.336905] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 2507.380544] Lustre: lustre-MDT0000: new disk, initializing [ 2507.410564] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 2507.418931] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 2509.090332] Lustre: DEBUG MARKER: oleg308-server.virtnet: executing set_default_debug -1 all [ 2514.684339] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 2514.726976] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 2514.787307] Lustre: 157743:0:(mgs_llog.c:1348:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 2514.829621] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [ 2514.833263] Lustre: Skipped 1 previous similar message [ 2514.884890] Lustre: lustre-MDT0001: new disk, initializing [ 2514.930581] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 2514.951071] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [ 2514.959528] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 2516.703750] Lustre: DEBUG MARKER: oleg308-server.virtnet: executing set_default_debug -1 all [ 2520.652787] Lustre: DEBUG MARKER: oleg308-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 2522.451279] Lustre: DEBUG MARKER: oleg308-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 2525.657920] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 2525.682622] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 2525.767534] Lustre: lustre-OST0000: new disk, initializing [ 2525.769159] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 2525.793644] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 2528.363189] Lustre: DEBUG MARKER: oleg308-server.virtnet: executing set_default_debug -1 all [ 2531.339355] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [ 2531.342416] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 2531.387331] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 2532.158894] Lustre: DEBUG MARKER: oleg308-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 2536.420892] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2536.422244] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 2536.429558] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 2536.433562] LustreError: Skipped 1 previous similar message [ 2538.413910] LustreError: 160330:0:(obd_class.h:479:obd_check_dev()) Device 19 not setup [ 2538.447842] Lustre: server umount lustre-OST0000 complete [ 2540.512509] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 2540.517199] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2540.524383] Lustre: Skipped 1 previous similar message [ 2540.527893] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 2540.532438] Lustre: Skipped 1 previous similar message [ 2541.538166] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 2541.540246] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2542.034412] LustreError: 160530:0:(obd_class.h:479:obd_check_dev()) Device 22 not setup [ 2542.037847] LustreError: 160530:0:(obd_class.h:479:obd_check_dev()) Skipped 3 previous similar messages [ 2542.073997] Lustre: server umount lustre-MDT0000 complete [ 2543.941594] LustreError: 157734:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1767387524 with bad export cookie 13868315568449672150 [ 2543.949476] LustreError: MGC192.168.203.108@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 2543.951169] LustreError: 157734:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 2544.046424] LustreError: 160733:0:(obd_class.h:479:obd_check_dev()) Device 21 not setup [ 2544.049660] LustreError: 160733:0:(obd_class.h:479:obd_check_dev()) Skipped 5 previous similar messages [ 2544.134363] Lustre: server umount lustre-MDT0001 complete [ 2548.903479] Lustre: DEBUG MARKER: oleg308-server.virtnet: executing unload_modules_local [ 2550.492674] Key type lgssc unregistered [ 2550.672674] LNet: 161266:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 2550.677873] LNetError: 161266:0:(acceptor.c:264:lnet_acceptor_remove_socket()) Interface ens2 not found [ 2550.686874] LNet: Removed LNI 192.168.203.108@tcp [ 2551.170930] Key type .llcrypt unregistered [ 2551.173495] Key type ._llcrypt unregistered [ 2563.493272] Key type ._llcrypt registered [ 2563.495109] Key type .llcrypt registered [ 2563.557964] Lustre: DEBUG MARKER: oleg308-server.virtnet: executing load_modules_local [ 2564.089346] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 2564.109804] alg: No test for adler32 (adler32-zlib) [ 2565.008921] Lustre: Lustre: Build Version: 2.17.0_RC4_2_g45eb11f [ 2565.120404] LNet: Added LNI 192.168.203.108@tcp [8/256/0/180] [ 2566.719292] Key type lgssc registered [ 2567.172778] Lustre: Echo OBD driver; http://www.lustre.org/ [ 2570.881058] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 2570.892564] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 2572.138540] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 2573.845439] Lustre: DEBUG MARKER: oleg308-server.virtnet: executing set_default_debug -1 all [ 2577.837022] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 2578.023841] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 2579.901901] Lustre: DEBUG MARKER: oleg308-server.virtnet: executing set_default_debug -1 all [ 2582.502200] Lustre: DEBUG MARKER: oleg308-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 2584.101542] Lustre: DEBUG MARKER: oleg308-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 2587.565893] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 2587.778561] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 2590.064242] Lustre: DEBUG MARKER: oleg308-server.virtnet: executing set_default_debug -1 all [ 2592.464755] Lustre: DEBUG MARKER: oleg308-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 2597.041962] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:3 to 0x280000401:33) [ 2600.428615] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 2600.441453] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2600.451553] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 2600.457338] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2600.464876] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 2600.837129] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2601.172111] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 2601.174168] Lustre: Skipped 1 previous similar message [ 2601.181544] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2601.187776] Lustre: Skipped 1 previous similar message [ 2601.190162] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 2601.191777] Lustre: Skipped 1 previous similar message [ 2601.193048] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2601.199533] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 2601.201870] Lustre: Skipped 1 previous similar message [ 2601.511528] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2601.889666] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2602.329596] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 2602.334554] Lustre: Skipped 2 previous similar messages [ 2602.344181] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2602.351973] Lustre: Skipped 2 previous similar messages [ 2602.355220] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 2602.359761] Lustre: Skipped 2 previous similar messages [ 2602.362515] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2602.369384] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 2602.373315] Lustre: Skipped 2 previous similar messages [ 2602.814445] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2603.270150] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2603.750304] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2604.197554] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2604.694102] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 2604.698277] Lustre: Skipped 4 previous similar messages [ 2604.707844] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2604.718130] Lustre: Skipped 4 previous similar messages [ 2604.722074] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 2604.726399] Lustre: Skipped 4 previous similar messages [ 2604.729294] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2604.734647] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 2604.741720] Lustre: Skipped 4 previous similar messages [ 2605.192319] LustreError: 162660:0:(osp_precreate.c:651:osp_precreate_send()) lustre-OST0000-osc-MDT0000: can't precreate: rc = -5 [ 2605.197778] LustreError: 162660:0:(osp_precreate.c:1355:osp_precreate_thread()) lustre-OST0000-osc-MDT0000: cannot precreate objects: rc = -5 [ 2605.199622] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2605.650713] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2606.128252] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2606.605914] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2607.069740] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2607.519784] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2607.987518] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2608.373457] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2608.711388] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 2608.712958] Lustre: Skipped 8 previous similar messages [ 2608.717733] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2608.720645] Lustre: Skipped 8 previous similar messages [ 2608.722051] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 2608.723745] Lustre: Skipped 8 previous similar messages [ 2608.724970] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2609.143641] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2609.146969] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 2609.151137] Lustre: Skipped 9 previous similar messages [ 2609.571457] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2609.978572] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2610.402821] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2610.816543] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2611.241160] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2611.668512] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2612.092662] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2612.511481] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2613.001925] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2613.457100] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2613.901518] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2614.366181] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2614.790888] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2615.240769] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2615.714179] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2616.127716] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2616.587138] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2617.065141] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 2617.070702] Lustre: Skipped 18 previous similar messages [ 2617.080978] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2617.088994] Lustre: Skipped 18 previous similar messages [ 2617.092280] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 2617.096390] Lustre: Skipped 18 previous similar messages [ 2617.099338] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2617.563592] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2617.570616] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 2617.574318] Lustre: Skipped 18 previous similar messages [ 2618.037964] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2618.533723] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2618.994880] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2619.429691] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2619.906189] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2620.365793] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2620.824318] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2621.313395] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2621.776672] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2622.220948] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2622.721541] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2623.169741] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2623.637341] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2624.120230] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2624.549118] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2624.992252] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2625.459590] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2625.883603] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2626.320714] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2626.760470] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2627.225335] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2627.671598] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2628.135616] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2628.576605] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2629.030283] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2629.490269] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2629.831437] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2630.195705] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2630.565139] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2630.978564] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2631.346950] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2631.681594] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2632.068798] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2632.467976] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2632.946774] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2633.392719] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 2633.396955] Lustre: Skipped 36 previous similar messages [ 2633.406936] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2633.414550] Lustre: Skipped 36 previous similar messages [ 2633.417415] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 2633.421420] Lustre: Skipped 36 previous similar messages [ 2633.424238] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2633.875233] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2633.882062] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 2633.885720] Lustre: Skipped 36 previous similar messages [ 2634.348954] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2634.808231] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2635.247695] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2635.725111] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2636.187489] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2636.637736] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2637.099777] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2637.544139] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2637.989490] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2638.443709] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2638.882081] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2639.331966] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2639.810383] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2640.262966] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2640.664994] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2641.126762] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2641.578389] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2642.034753] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2642.498771] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2642.928146] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2643.380533] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2643.816783] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2644.223623] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2644.682710] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2645.145484] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2645.563145] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2645.981631] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2646.422295] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2646.872296] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2647.334677] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2647.813870] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2648.268953] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2648.739281] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2649.239110] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2649.686078] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2650.071987] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2650.556457] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2651.016106] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2651.470615] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2651.933419] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2652.372099] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2652.837599] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2653.289108] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2653.756534] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2654.220210] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2654.643334] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2655.066298] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2655.520045] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2656.003273] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2656.467623] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2656.922971] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2657.400056] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2657.859566] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2658.328956] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2658.810448] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2659.266711] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2659.717156] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2660.129437] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2660.565768] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2660.943507] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2661.389295] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2661.842225] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2662.289800] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2662.771591] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2663.221466] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2663.670553] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2664.098084] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2664.460403] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2664.872911] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2665.272531] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2665.664413] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 2665.668438] Lustre: Skipped 71 previous similar messages [ 2665.676709] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2665.683019] Lustre: Skipped 71 previous similar messages [ 2665.685272] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 2665.689027] Lustre: Skipped 71 previous similar messages [ 2665.691470] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2666.145051] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2666.153211] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 2666.157736] Lustre: Skipped 71 previous similar messages [ 2666.635296] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2667.115159] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2667.578322] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2668.058598] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2668.520446] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2668.978060] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2669.464610] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2669.918737] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2670.385330] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2670.880866] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2671.340174] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2671.813303] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2672.304248] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2672.759309] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2673.173586] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2673.653429] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2674.113703] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2674.583981] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2675.053751] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2675.470544] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2675.904905] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2676.398666] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2676.846334] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2677.293615] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2677.784226] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2678.219905] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2678.670513] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2679.128472] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2679.582307] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2680.032322] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2680.470830] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2680.943611] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2681.419787] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2681.943215] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2682.362060] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2682.780260] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2683.251502] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2683.705204] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2684.159667] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2684.635684] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2685.081348] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2685.567943] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2686.063752] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2686.524315] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2686.979804] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2687.469396] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2687.930981] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2688.399921] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2688.875445] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2689.349585] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2689.788299] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2690.234060] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2690.665297] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2691.121236] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2691.551965] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2691.937487] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2692.326925] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2692.719504] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2693.150026] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2693.545948] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2693.976533] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2694.393482] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2694.802972] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2695.219419] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2695.640133] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2696.055700] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2696.506627] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2696.958685] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2697.383655] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2697.828247] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2698.266210] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2698.702109] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2699.153462] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2699.571808] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2699.986297] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2700.295932] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2700.675443] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2701.067308] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2701.401263] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2701.770187] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2702.174256] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2702.615427] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2703.014909] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2703.441987] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2703.864984] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2704.266514] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2704.669989] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2705.107862] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2705.543142] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2705.963610] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2706.424517] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2706.838624] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2707.224891] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2707.602554] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2707.982592] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2708.396122] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2708.813536] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2709.195164] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2709.579116] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2709.949566] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2710.337132] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2710.716067] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2711.098028] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2711.424619] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2711.790489] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2712.156608] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2712.499231] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2712.900594] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2713.323614] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2713.712928] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2714.125759] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2714.489585] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2714.814977] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2715.207571] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2715.604539] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2716.000474] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2716.378242] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2716.738244] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2717.126259] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2717.580063] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2718.069579] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2718.548142] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2719.005771] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2719.487252] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2719.949731] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2720.402179] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2720.889410] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2721.326791] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2721.781468] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2722.231478] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2722.680521] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2723.136502] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2723.594607] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2723.996512] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2724.312381] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2724.655437] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2725.113362] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2725.571004] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2726.065811] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2726.486750] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2726.949454] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2727.436968] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2727.899863] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2728.340848] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2728.841480] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2729.301604] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2729.756808] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 2729.762202] Lustre: Skipped 147 previous similar messages [ 2729.771835] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2729.779660] Lustre: Skipped 147 previous similar messages [ 2729.783087] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 2729.787139] Lustre: Skipped 147 previous similar messages [ 2729.789973] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2730.266343] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2730.273328] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 2730.275901] Lustre: Skipped 147 previous similar messages [ 2730.715390] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2731.217570] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2731.679382] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2732.116969] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2732.577499] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2733.086551] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2733.566388] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2734.043903] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2734.537575] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2734.984314] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2735.460273] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2735.963623] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2736.412710] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2736.861625] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2737.348334] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2737.816917] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2738.232528] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2738.745648] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2739.213541] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2739.662187] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2740.154496] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2740.590987] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2741.045478] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2741.539672] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2741.999592] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2742.463037] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2742.906773] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2743.354152] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2743.820785] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2744.290472] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2744.622955] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2744.967880] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2745.379197] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2745.844923] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2746.307604] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2746.758717] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2747.165453] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2747.636656] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2748.108216] LustreError: 162660:0:(osp_precreate.c:651:osp_precreate_send()) lustre-OST0000-osc-MDT0000: can't precreate: rc = -5 [ 2748.114138] LustreError: 162660:0:(osp_precreate.c:1355:osp_precreate_thread()) lustre-OST0000-osc-MDT0000: cannot precreate objects: rc = -5 [ 2748.115059] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2748.569894] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2749.024476] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2749.522933] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2749.990711] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2750.469206] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2750.919364] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2751.392746] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2751.814829] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2752.141971] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2752.485501] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2752.874097] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2753.348759] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2753.826234] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2754.265907] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2754.766449] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2755.201262] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2755.610359] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2756.017447] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2756.441300] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2756.865900] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2757.273587] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2757.627478] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2758.041163] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2758.457335] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2758.869830] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2759.286204] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2759.728488] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2760.144638] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2760.591992] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2761.045696] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2761.490665] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2761.929165] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2762.382888] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2762.844760] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2763.318639] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2763.786383] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2764.184314] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2764.622607] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2765.114417] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2765.580690] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2766.034794] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2766.530327] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2766.950645] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2767.331777] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2767.809774] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2768.249422] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2768.689567] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2769.142407] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2769.551612] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2769.931825] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2770.320503] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2770.680924] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2771.092506] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2771.496251] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2771.912335] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2772.361473] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2772.852403] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2773.308693] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2773.768771] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2774.251655] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2774.701095] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2775.157272] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2775.638699] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2776.093794] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2776.573465] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2777.039293] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2777.464311] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2777.891306] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2778.338681] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2778.782854] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2779.214484] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2779.699530] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2780.150926] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2780.604974] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2781.098535] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2781.554695] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2782.007282] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2782.494644] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2782.946857] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2783.399556] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2783.876549] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2784.333094] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2784.785386] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2785.252459] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2785.732993] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2786.176737] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2786.653941] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2787.100751] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2787.553270] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2788.031292] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2788.480825] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2788.926177] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2789.396695] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2789.840516] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2790.278657] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2790.771153] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2791.239956] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2791.698703] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2792.186665] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2792.645936] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2793.110797] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2793.591372] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2794.056319] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2794.486668] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2795.000226] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2795.440745] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2795.897394] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2796.344397] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2796.782467] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2797.231570] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2797.666849] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2798.117442] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2798.571236] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2799.060680] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2799.458097] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2799.856269] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2800.232804] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2800.610477] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2801.072281] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2801.570501] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2802.030574] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2802.476599] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2802.954057] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2803.404372] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2803.881130] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2804.329662] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2804.773647] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2805.222851] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2805.657954] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2806.094900] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2806.526924] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2807.009948] LustreError: 162660:0:(osp_precreate.c:651:osp_precreate_send()) lustre-OST0000-osc-MDT0000: can't precreate: rc = -5 [ 2807.015729] LustreError: 162660:0:(osp_precreate.c:1355:osp_precreate_thread()) lustre-OST0000-osc-MDT0000: cannot precreate objects: rc = -5 [ 2807.017607] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2807.473874] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2807.930030] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2808.414800] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2808.895275] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2809.386668] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2809.882815] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2810.334951] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2810.796396] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2811.294645] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2811.723391] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2812.095881] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2812.545913] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2813.024185] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2813.480050] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2813.992220] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2814.448223] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2814.918155] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2815.401360] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2815.833567] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2816.289397] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2816.753723] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2817.220483] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2817.665750] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2818.141465] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2818.565780] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2819.016172] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2819.453949] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2819.878475] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2820.321371] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2820.785252] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2821.240477] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2821.683264] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2822.084809] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2822.486068] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2822.932665] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2823.423520] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2823.826089] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2824.244101] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2824.643651] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2825.082096] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2825.508528] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2825.952667] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2826.394314] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2826.825328] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2827.284640] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2827.712488] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2828.135816] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2828.536397] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2828.946057] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2829.333972] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2829.768432] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2830.177164] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2830.554937] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2830.824480] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2831.091918] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2831.357628] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2831.713508] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2832.163273] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2832.612629] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2833.054867] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2833.450767] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2833.893751] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2834.348658] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2834.806326] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2835.250575] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2835.675580] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2836.122600] LustreError: 162660:0:(osp_precreate.c:651:osp_precreate_send()) lustre-OST0000-osc-MDT0000: can't precreate: rc = -5 [ 2836.128962] LustreError: 162660:0:(osp_precreate.c:1355:osp_precreate_thread()) lustre-OST0000-osc-MDT0000: cannot precreate objects: rc = -5 [ 2836.130107] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2836.569145] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2837.017286] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2837.475131] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2837.916037] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2838.341762] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2838.809085] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2839.252351] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2839.689099] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2840.162767] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2840.615845] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2841.054500] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2841.476143] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2841.918744] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2842.358633] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2842.817057] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2843.254549] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2843.689800] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2844.149604] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2844.594552] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2845.037345] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2845.492240] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2845.934120] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2846.385930] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2846.852305] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2847.305703] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2847.759148] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2848.216397] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2848.649299] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2849.097395] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2849.570976] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2850.018077] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2850.455297] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2850.902531] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2851.329305] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2851.778865] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2852.248132] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2852.689578] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2853.127672] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2853.593712] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2854.016753] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2854.460866] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2854.928506] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2855.351559] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2855.773525] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2856.202181] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2856.663342] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2857.084903] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2857.536647] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2857.974481] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 2857.978584] Lustre: Skipped 287 previous similar messages [ 2857.987545] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2857.994777] Lustre: Skipped 287 previous similar messages [ 2857.997678] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 2858.001734] Lustre: Skipped 287 previous similar messages [ 2858.004535] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2858.436539] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2858.443461] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 2858.447414] Lustre: Skipped 287 previous similar messages [ 2858.887699] LustreError: 162660:0:(osp_precreate.c:651:osp_precreate_send()) lustre-OST0000-osc-MDT0000: can't precreate: rc = -5 [ 2858.894079] LustreError: 162660:0:(osp_precreate.c:1355:osp_precreate_thread()) lustre-OST0000-osc-MDT0000: cannot precreate objects: rc = -5 [ 2858.895444] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2859.339625] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2859.788298] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2860.259522] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2860.700095] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2861.129364] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2861.597546] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2862.043855] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2862.487925] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2862.948922] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2863.390696] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2863.839254] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2864.295528] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2864.722571] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2865.165209] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2865.632684] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2866.061591] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2866.505504] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2866.968049] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2867.391984] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2867.836669] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2868.302921] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2868.736311] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2869.177625] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2869.626955] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2870.062069] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2870.508819] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2870.979591] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2871.423411] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2871.859540] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2872.273430] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2872.565577] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2872.885104] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2873.219986] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2873.585355] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2873.951886] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2874.395978] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2874.847369] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2875.269909] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2875.682946] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2876.134547] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2876.554510] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2876.983495] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2877.446382] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2877.881808] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2878.327561] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2878.767463] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2879.207635] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2879.645945] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2880.104306] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2880.540434] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2880.973916] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2881.421774] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2881.855767] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2882.295398] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2882.708862] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2883.145955] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2883.582715] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2884.035175] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2884.388846] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2884.822072] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2885.287247] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2885.697228] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2886.126609] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2886.582135] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2887.047723] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2887.470538] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2887.930671] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2888.370145] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2888.799284] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2889.204491] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2889.577373] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2889.955526] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2890.339632] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2890.720431] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2891.098427] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2891.477669] LustreError: 162660:0:(osp_precreate.c:651:osp_precreate_send()) lustre-OST0000-osc-MDT0000: can't precreate: rc = -5 [ 2891.480917] LustreError: 162660:0:(osp_precreate.c:1355:osp_precreate_thread()) lustre-OST0000-osc-MDT0000: cannot precreate objects: rc = -5 [ 2891.482403] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2891.849429] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2892.209838] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2892.597442] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2892.972350] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2893.341966] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2893.751447] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2894.183417] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2894.608413] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2895.075405] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2895.505366] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2895.928196] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2896.379793] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2896.818536] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2897.174257] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2897.596371] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2898.017175] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2898.450711] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2898.824810] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2899.216650] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2899.660979] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2900.127313] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2900.574834] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2901.022511] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2901.491870] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2901.935306] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2902.374464] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2902.826544] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2903.269486] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2903.709100] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2904.175352] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2904.616920] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2905.056727] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2905.499520] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2905.935534] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2906.363967] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2906.813226] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2907.252272] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2907.692454] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2908.137319] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2908.583335] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2909.025324] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2909.503717] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2909.941305] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2910.403526] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2910.884406] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2911.316888] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2911.777898] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2912.239180] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2912.699296] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2913.134485] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2913.617633] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2914.088566] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2914.560262] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2915.014179] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2915.427551] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2915.894831] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2916.364319] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2916.831375] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2917.312350] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2917.779510] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2918.221244] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2918.656079] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2919.128035] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2919.578401] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2920.014467] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2920.464397] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2920.910811] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2921.360365] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2921.841560] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2922.284857] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2922.727272] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2923.197153] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2923.644987] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2924.092342] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2924.529507] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2924.954710] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2925.386388] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2925.841317] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2926.294714] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2926.752175] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2927.212431] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2927.656594] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2928.106573] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2928.565329] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2929.021371] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2929.475313] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2929.960405] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2930.414454] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2930.868538] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2931.318307] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2931.773627] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2932.227534] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2932.618147] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2932.963960] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2933.375427] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2933.805514] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2934.213979] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2934.666912] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2935.149251] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2935.609559] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2936.065497] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2936.503880] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2936.953516] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2937.392211] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2937.867468] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2938.287626] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2938.722899] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2939.192230] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2939.652477] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2940.101422] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2940.574038] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2941.012115] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2941.455257] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2941.909109] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2942.347981] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2942.772855] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2943.219484] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2943.640151] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2944.105213] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2944.561416] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2944.969049] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2945.374127] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2945.787155] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2946.188564] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2946.585383] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2946.980040] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2947.410082] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2947.845859] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2948.304458] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2948.740191] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2949.175226] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2949.635796] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2950.074231] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2950.495835] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2950.961985] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2951.400463] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2951.868790] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2952.305446] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2952.715356] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2953.160262] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2953.630948] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2954.083848] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2954.522665] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2954.970108] LustreError: 162660:0:(osp_precreate.c:651:osp_precreate_send()) lustre-OST0000-osc-MDT0000: can't precreate: rc = -5 [ 2954.977398] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2954.978700] LustreError: 162660:0:(osp_precreate.c:1355:osp_precreate_thread()) lustre-OST0000-osc-MDT0000: cannot precreate objects: rc = -5 [ 2955.424809] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2955.852118] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2956.301092] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2956.735236] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2957.187484] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2957.640725] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2958.083339] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2958.525724] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2958.976641] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2959.417137] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2959.853317] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2960.300101] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2960.744636] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2961.172957] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2961.642267] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2962.070185] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2962.509948] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2962.974313] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2963.402094] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2963.855945] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2964.317750] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2964.761310] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2965.184312] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2965.650610] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2966.118527] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2966.562965] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2967.036417] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2967.490805] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2967.939699] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2968.421463] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2968.873726] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2969.305263] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2969.769657] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2970.203499] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2970.660789] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2971.113874] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2971.494390] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2971.885390] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2972.320882] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2972.753885] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2973.186349] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2973.656992] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2974.087412] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2974.528289] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2975.016381] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2975.462115] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2975.899450] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2976.360951] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2976.810492] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2977.244669] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2977.689889] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2978.135727] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2978.561970] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2979.022533] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2979.459137] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2979.906766] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2980.372573] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2980.805633] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2981.250110] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2981.714514] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2982.131376] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2982.566070] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2983.026712] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2983.465954] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2983.910767] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2984.384298] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2984.817481] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2985.262662] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2985.734466] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2986.178178] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2986.591553] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2987.057313] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2987.484457] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2987.925239] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2988.396975] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2988.843108] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2989.285889] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2989.759406] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2990.195102] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2990.627967] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2991.061119] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2991.409148] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2991.844900] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2992.188329] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2992.610292] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2992.928423] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2993.363773] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2993.785194] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2994.224747] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2994.686612] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2995.132862] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2995.502400] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2995.935859] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2996.336166] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2996.705092] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2997.078269] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2997.439010] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2997.816201] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2998.183813] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2998.567363] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2999.004647] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2999.442522] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2999.870465] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3000.279676] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3000.731812] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3001.148784] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3001.594514] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3002.073320] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3002.512745] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3002.953494] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3003.399287] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3003.853675] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3004.300261] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3004.769504] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3005.224037] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3005.661295] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3006.135707] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3006.491735] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3006.843040] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3007.192904] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3007.591237] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3007.985621] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3008.403754] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3008.841653] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3009.298475] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3009.681773] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3010.087662] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3010.471844] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3010.854530] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3011.200461] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3011.552907] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3011.975923] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3012.297522] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3012.614973] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3013.048729] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3013.391181] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3013.794546] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3014.254864] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3014.703454] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3015.157290] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3015.611739] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3016.049513] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3016.340574] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3016.625136] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3017.044539] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3017.480514] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3017.855534] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3018.311118] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3018.694174] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3019.121876] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3019.539929] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3020.010373] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3020.312287] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3020.588963] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3020.880852] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3021.176116] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3021.560934] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3022.003950] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3022.478346] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3022.912887] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3023.354172] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3023.789492] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3024.236585] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3024.682625] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3025.158604] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3025.597096] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3026.016572] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3026.437123] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3026.881407] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3027.263159] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3027.665711] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3027.998718] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3028.296480] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3028.659836] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3029.151472] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3029.581479] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3030.019246] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3030.484331] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3030.936957] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3031.352633] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3031.829551] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3032.209206] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3032.609454] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3033.051248] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3033.505893] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3033.940643] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3034.399331] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3034.845042] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3035.281465] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3035.760994] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3036.182136] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3036.585981] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3037.002235] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3037.436464] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3159.308481] LustreError: 164238:0:(ofd_dev.c:1780:ofd_create_hdl()) lustre-OST0000: unable to precreate: rc = -28 [ 3159.314316] LustreError: 162660:0:(osp_precreate.c:651:osp_precreate_send()) lustre-OST0000-osc-MDT0000: can't precreate: rc = -28 [ 3264.992419] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 3264.997135] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 3265.004218] Lustre: Skipped 416 previous similar messages [ 3265.007584] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 3267.055288] LustreError: lustre-OST0000-osc-MDT0000: operation ost_destroy to node 0@lo failed: rc = -107 [ 3267.058848] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 3267.061283] LustreError: Skipped 4 previous similar messages [ 3268.067142] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 3268.070838] Lustre: Skipped 1 previous similar message [ 3269.760189] LustreError: 216428:0:(obd_class.h:479:obd_check_dev()) Device 21 not setup [ 3269.858866] Lustre: server umount lustre-OST0000 complete [ 3273.186601] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 3273.189306] Lustre: Skipped 1 previous similar message [ 3277.841433] LustreError: 216629:0:(obd_class.h:479:obd_check_dev()) Device 9 not setup [ 3277.846232] LustreError: 216629:0:(obd_class.h:479:obd_check_dev()) Skipped 3 previous similar messages [ 3277.945152] Lustre: server umount lustre-MDT0000 complete [ 3278.305722] LustreError: 162641:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 3278.315203] LustreError: 162641:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 3 previous similar messages [ 3279.739842] LustreError: 162622:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1767388260 with bad export cookie 6914674563460945197 [ 3279.745849] LustreError: MGC192.168.203.108@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 3279.746666] LustreError: 162622:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) Skipped 3 previous similar messages [ 3279.854956] LustreError: 216832:0:(obd_class.h:479:obd_check_dev()) Device 16 not setup [ 3279.858414] LustreError: 216832:0:(obd_class.h:479:obd_check_dev()) Skipped 5 previous similar messages [ 3279.915275] Lustre: server umount lustre-MDT0001 complete [ 3284.677663] Lustre: DEBUG MARKER: oleg308-server.virtnet: executing unload_modules_local [ 3286.057188] Key type lgssc unregistered [ 3286.219469] LNet: 217362:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 3286.224176] LNetError: 217362:0:(acceptor.c:264:lnet_acceptor_remove_socket()) Interface ens2 not found [ 3286.232573] LNet: Removed LNI 192.168.203.108@tcp [ 3286.658468] Key type .llcrypt unregistered [ 3286.659895] Key type ._llcrypt unregistered [ 3298.165190] Key type ._llcrypt registered [ 3298.167249] Key type .llcrypt registered [ 3298.216363] Lustre: DEBUG MARKER: oleg308-server.virtnet: executing load_modules_local [ 3298.548869] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 3298.591120] alg: No test for adler32 (adler32-zlib) [ 3299.491077] Lustre: Lustre: Build Version: 2.17.0_RC4_2_g45eb11f [ 3299.598250] LNet: Added LNI 192.168.203.108@tcp [8/256/0/180] [ 3301.191155] Key type lgssc registered [ 3301.817672] Lustre: Echo OBD driver; http://www.lustre.org/ [ 3305.607637] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 3305.618431] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 3306.842107] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 3308.653953] Lustre: DEBUG MARKER: oleg308-server.virtnet: executing set_default_debug -1 all [ 3312.730111] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 3312.926540] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 3314.631833] Lustre: DEBUG MARKER: oleg308-server.virtnet: executing set_default_debug -1 all [ 3317.082631] Lustre: DEBUG MARKER: oleg308-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 3318.844669] Lustre: DEBUG MARKER: oleg308-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 3322.194720] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 3322.368833] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 3324.903564] Lustre: DEBUG MARKER: oleg308-server.virtnet: executing set_default_debug -1 all [ 3327.481929] Lustre: DEBUG MARKER: oleg308-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 3331.631577] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:49666 to 0x280000401:49697) [ 3334.909252] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 3335.784914] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 3335.793901] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 3335.798736] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3335.805665] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 3335.805699] LustreError: 218754:0:(osp_precreate.c:1263:osp_precreate_thread()) cfs_fail_timeout id 2107 sleeping for 20000ms [ 3336.675832] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 3336.677902] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 3336.680768] Lustre: Skipped 1 previous similar message [ 3336.686855] Lustre: Skipped 1 previous similar message [ 3338.183788] Lustre: lustre-OST0000: Not available for connect from 192.168.203.8@tcp (stopping) [ 3338.744199] LustreError: 221133:0:(obd_class.h:479:obd_check_dev()) Device 21 not setup [ 3338.812187] Lustre: server umount lustre-OST0000 complete [ 3355.887064] LustreError: 218754:0:(osp_precreate.c:1263:osp_precreate_thread()) cfs_fail_timeout id 2107 awake [ 3367.985797] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 3368.131340] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 3370.291137] Lustre: DEBUG MARKER: oleg308-server.virtnet: executing set_default_debug -1 all [ 3372.541017] Lustre: DEBUG MARKER: oleg308-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 3376.359134] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3376.364649] LustreError: lustre-OST0000-osc-MDT0001: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3376.364989] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 3376.375126] Lustre: Skipped 1 previous similar message [ 3379.577843] Lustre: DEBUG MARKER: osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid in FULL state after 6 sec [ 3381.285514] Lustre: DEBUG MARKER: oleg308-client.virtnet: executing wait_import_state (FULL|IDLE) osc.lustre-OST0000-osc-ffff9d4544be5000.ost_server_uuid 50 [ 3381.984390] Lustre: DEBUG MARKER: osc.lustre-OST0000-osc-ffff9d4544be5000.ost_server_uuid in FULL state after 0 sec [ 3386.850074] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 3386.851692] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 3386.856106] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 3386.860352] Lustre: Skipped 1 previous similar message [ 3386.867954] Lustre: Skipped 1 previous similar message [ 3389.297135] LustreError: 222451:0:(obd_class.h:479:obd_check_dev()) Device 21 not setup [ 3389.302972] LustreError: 222451:0:(obd_class.h:479:obd_check_dev()) Skipped 3 previous similar messages [ 3389.362265] Lustre: server umount lustre-OST0000 complete [ 3391.969257] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 3391.970474] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 3391.977910] Lustre: Skipped 1 previous similar message [ 3391.987363] Lustre: Skipped 1 previous similar message [ 3397.089219] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 3397.094054] Lustre: Skipped 1 previous similar message [ 3397.201962] LustreError: 222653:0:(obd_class.h:479:obd_check_dev()) Device 9 not setup [ 3397.205807] LustreError: 222653:0:(obd_class.h:479:obd_check_dev()) Skipped 3 previous similar messages [ 3397.240078] Lustre: server umount lustre-MDT0000 complete [ 3398.865585] LustreError: 219454:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1767388379 with bad export cookie 14783920646355226182 [ 3398.869027] LustreError: 219454:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) Skipped 3 previous similar messages [ 3398.869883] LustreError: MGC192.168.203.108@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 3399.001574] Lustre: server umount lustre-MDT0001 complete [ 3403.392493] Lustre: DEBUG MARKER: oleg308-server.virtnet: executing unload_modules_local [ 3404.592760] Key type lgssc unregistered [ 3404.754571] LNet: 223386:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 3404.759432] LNetError: 223386:0:(acceptor.c:264:lnet_acceptor_remove_socket()) Interface ens2 not found [ 3404.769543] LNet: Removed LNI 192.168.203.108@tcp [ 3405.200432] Key type .llcrypt unregistered [ 3405.202728] Key type ._llcrypt unregistered [ 3419.567971] Key type ._llcrypt registered [ 3419.569146] Key type .llcrypt registered [ 3419.613361] Lustre: DEBUG MARKER: oleg308-server.virtnet: executing load_modules_local [ 3420.003499] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 3420.024070] alg: No test for adler32 (adler32-zlib) [ 3420.923670] Lustre: Lustre: Build Version: 2.17.0_RC4_2_g45eb11f [ 3421.039632] LNet: Added LNI 192.168.203.108@tcp [8/256/0/180] [ 3422.655176] Key type lgssc registered [ 3423.272439] Lustre: Echo OBD driver; http://www.lustre.org/ [ 3427.877928] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 3427.889662] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 3429.116595] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 3430.985482] Lustre: DEBUG MARKER: oleg308-server.virtnet: executing set_default_debug -1 all [ 3435.242314] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 3435.455265] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 3437.084905] Lustre: DEBUG MARKER: oleg308-server.virtnet: executing set_default_debug -1 all [ 3438.520440] Lustre: Modifying parameter general.debug_raw_pointers=Y in log params [ 3441.812289] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 3441.984129] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 3444.877568] Lustre: DEBUG MARKER: oleg308-server.virtnet: executing set_default_debug -1 all [ 3448.809059] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 3448.852322] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 3448.928850] Lustre: lustre-OST0001: new disk, initializing [ 3448.932396] Lustre: srv-lustre-OST0001: No data found on store. Initialize space. [ 3448.987567] Lustre: lustre-OST0001: Imperative Recovery not enabled, recovery window 60-180 [ 3451.238523] Lustre: DEBUG MARKER: oleg308-server.virtnet: executing set_default_debug -1 all [ 3452.569662] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x00000002c0000400-0x0000000300000400]:1:ost [ 3452.575574] Lustre: cli-lustre-OST0001-super: Allocated super-sequence [0x00000002c0000400-0x0000000300000400]:1:ost] [ 3452.635069] Lustre: lustre-OST0001-osc-MDT0000: update sequence from 0x100010000 to 0x2c0000401 [ 3453.683347] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:49699 to 0x280000401:49729) [ 3457.780832] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 3461.076670] Lustre: Setting parameter general.lod.*.mdt_hash=crush in log params [ 3474.404331] Lustre: lustre-MDT0000-lwp-OST0000: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 3474.408748] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 3474.411626] Lustre: Skipped 3 previous similar messages [ 3474.415338] Lustre: Skipped 2 previous similar messages [ 3479.380203] LustreError: 228432:0:(obd_class.h:479:obd_check_dev()) Device 28 not setup [ 3479.427835] Lustre: server umount lustre-MDT0000 complete [ 3479.524099] LustreError: 226171:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 3479.532546] LustreError: 226171:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 5 previous similar messages [ 3481.237920] LustreError: 225052:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1767388461 with bad export cookie 10318559997767286473 [ 3481.247863] LustreError: 225052:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) Skipped 3 previous similar messages [ 3481.247898] LustreError: MGC192.168.203.108@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 3481.350187] LustreError: 228634:0:(obd_class.h:479:obd_check_dev()) Device 27 not setup [ 3481.353945] LustreError: 228634:0:(obd_class.h:479:obd_check_dev()) Skipped 7 previous similar messages [ 3481.439351] Lustre: server umount lustre-MDT0001 complete [ 3493.882140] LustreError: 228835:0:(obd_class.h:479:obd_check_dev()) Device 21 not setup [ 3493.885995] LustreError: 228835:0:(obd_class.h:479:obd_check_dev()) Skipped 7 previous similar messages [ 3493.914705] Lustre: server umount lustre-OST0000 complete [ 3505.912801] LustreError: 229037:0:(obd_class.h:479:obd_check_dev()) Device 2 not setup [ 3505.916564] LustreError: 229037:0:(obd_class.h:479:obd_check_dev()) Skipped 3 previous similar messages [ 3505.987271] Lustre: server umount lustre-OST0001 complete [ 3507.546354] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro [ 3507.585645] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 3508.115076] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro [ 3508.154906] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 3508.696440] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: ,errors=remount-ro [ 3508.744092] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 3509.247777] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: ,errors=remount-ro [ 3509.291492] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 3516.013925] Lustre: DEBUG MARKER: oleg308-server.virtnet: executing load_modules_local [ 3520.420641] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 3520.618105] LustreError: 230445:0:(ldlm_lib.c:1179:target_handle_connect()) mylustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 3520.642069] Lustre: mylustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 3522.354824] Lustre: DEBUG MARKER: oleg308-server.virtnet: executing set_default_debug -1 all [ 3525.763365] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 3527.507397] Lustre: DEBUG MARKER: oleg308-server.virtnet: executing set_default_debug -1 all [ 3528.756857] Lustre: 231554:0:(mgs_llog.c:1348:mgs_modify_param()) MGS: modify general/debug_raw_pointers=Y (mode = 0) failed: rc = -17 [ 3531.642654] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 3531.831361] Lustre: mylustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 3531.835499] Lustre: Skipped 1 previous similar message [ 3534.249159] Lustre: DEBUG MARKER: oleg308-server.virtnet: executing set_default_debug -1 all [ 3535.911893] LustreError: 232453:0:(ldlm_lib.c:1179:target_handle_connect()) mylustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 3538.102488] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 3540.812576] Lustre: DEBUG MARKER: oleg308-server.virtnet: executing set_default_debug -1 all [ 3546.091957] Lustre: mylustre-OST0001: new connection from mylustre-MDT0000-mdtlov (cleaning up unused objects from 0x2c0000401:3 to 0x2c0000401:33) [ 3546.092248] Lustre: mylustre-OST0000: new connection from mylustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:49731 to 0x280000401:49761) [ 3550.119924] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 3551.895655] Lustre: 233403:0:(mgs_llog.c:1348:mgs_modify_param()) MGS: modify general/lod.*.mdt_hash=crush (mode = 0) failed: rc = -17 [ 3564.239912] LustreError: 233711:0:(obd_class.h:479:obd_check_dev()) Device 10 not setup [ 3564.243810] LustreError: 233711:0:(obd_class.h:479:obd_check_dev()) Skipped 5 previous similar messages [ 3564.280082] Lustre: server umount mylustre-MDT0000 complete [ 3566.019539] LustreError: 230420:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1767388546 with bad export cookie 10318559997767289266 [ 3566.027916] LustreError: MGC192.168.203.108@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 3566.029392] LustreError: 230420:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) Skipped 5 previous similar messages [ 3566.562466] Lustre: mylustre-MDT0001-lwp-OST0000: Connection to mylustre-MDT0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 3566.564760] LustreError: 233913:0:(ldlm_lib.c:1179:target_handle_connect()) mylustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 3566.565812] Lustre: mylustre-MDT0001: Not available for connect from 0@lo (stopping) [ 3566.569028] Lustre: Skipped 3 previous similar messages [ 3566.586648] LustreError: 233913:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 1 previous similar message [ 3568.214763] Lustre: server umount mylustre-MDT0001 complete [ 3570.160731] Lustre: server umount mylustre-OST0000 complete [ 3572.007802] Lustre: server umount mylustre-OST0001 complete [ 3573.529677] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro [ 3573.571126] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 3574.096645] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro [ 3574.145113] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 3574.666065] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: ,errors=remount-ro [ 3574.707649] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 3575.230091] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: ,errors=remount-ro [ 3575.284065] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 3582.025878] Lustre: DEBUG MARKER: oleg308-server.virtnet: executing load_modules_local [ 3586.046169] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 3586.265490] LustreError: 235721:0:(ldlm_lib.c:1179:target_handle_connect()) tfs-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 3586.297850] Lustre: tfs-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 3586.300335] Lustre: Skipped 1 previous similar message [ 3588.038679] Lustre: DEBUG MARKER: oleg308-server.virtnet: executing set_default_debug -1 all [ 3591.935992] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 3593.778893] Lustre: DEBUG MARKER: oleg308-server.virtnet: executing set_default_debug -1 all [ 3595.155605] Lustre: 236832:0:(mgs_llog.c:1348:mgs_modify_param()) MGS: modify general/debug_raw_pointers=Y (mode = 0) failed: rc = -17 [ 3597.865996] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 3600.549635] Lustre: DEBUG MARKER: oleg308-server.virtnet: executing set_default_debug -1 all [ 3602.150358] LustreError: 237188:0:(ldlm_lib.c:1179:target_handle_connect()) tfs-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 3602.163359] LustreError: 237188:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 1 previous similar message [ 3604.712810] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 3607.254320] Lustre: DEBUG MARKER: oleg308-server.virtnet: executing set_default_debug -1 all [ 3610.793759] Lustre: tfs-OST0001: new connection from tfs-MDT0000-mdtlov (cleaning up unused objects from 0x2c0000401:35 to 0x2c0000401:65) [ 3610.793784] Lustre: tfs-OST0000: new connection from tfs-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:49731 to 0x280000401:49793) [ 3616.812934] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 3618.639895] Lustre: 238678:0:(mgs_llog.c:1348:mgs_modify_param()) MGS: modify general/lod.*.mdt_hash=crush (mode = 0) failed: rc = -17 [ 3626.469290] Lustre: tfs-MDT0000-lwp-OST0001: Connection to tfs-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 3626.472555] Lustre: tfs-MDT0000: Not available for connect from 0@lo (stopping) [ 3626.478475] Lustre: Skipped 3 previous similar messages [ 3626.485379] Lustre: Skipped 4 previous similar messages [ 3631.586797] Lustre: tfs-MDT0000: Not available for connect from 0@lo (stopping) [ 3631.590308] Lustre: Skipped 2 previous similar messages [ 3632.533344] LustreError: 238928:0:(obd_class.h:479:obd_check_dev()) Device 10 not setup [ 3632.536580] LustreError: 238928:0:(obd_class.h:479:obd_check_dev()) Skipped 25 previous similar messages [ 3632.570151] Lustre: server umount tfs-MDT0000 complete [ 3634.362539] LustreError: 238680:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1767388615 with bad export cookie 10318559997767291156 [ 3634.370410] LustreError: MGC192.168.203.108@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 3634.540539] Lustre: server umount tfs-MDT0001 complete [ 3646.932680] Lustre: server umount tfs-OST0000 complete [ 3659.000825] LustreError: 239532:0:(obd_class.h:479:obd_check_dev()) Device 2 not setup [ 3659.004621] LustreError: 239532:0:(obd_class.h:479:obd_check_dev()) Skipped 19 previous similar messages [ 3659.070838] Lustre: server umount tfs-OST0001 complete [ 3660.631867] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro [ 3660.678087] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 3661.206654] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro [ 3661.252702] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 3661.668678] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: ,errors=remount-ro [ 3661.693155] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 3662.019272] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: ,errors=remount-ro [ 3662.039734] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 3667.938933] Lustre: DEBUG MARKER: oleg308-server.virtnet: executing load_modules_local [ 3672.825549] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 3673.047439] LustreError: 240939:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 3673.085058] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 3673.088319] Lustre: Skipped 3 previous similar messages [ 3674.739820] Lustre: DEBUG MARKER: oleg308-server.virtnet: executing set_default_debug -1 all [ 3678.604258] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 3680.548788] Lustre: DEBUG MARKER: oleg308-server.virtnet: executing set_default_debug -1 all [ 3681.986585] Lustre: 242050:0:(mgs_llog.c:1348:mgs_modify_param()) MGS: modify general/debug_raw_pointers=Y (mode = 0) failed: rc = -17 [ 3685.190716] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 3687.957324] Lustre: DEBUG MARKER: oleg308-server.virtnet: executing set_default_debug -1 all [ 3692.074625] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 3694.647871] Lustre: DEBUG MARKER: oleg308-server.virtnet: executing set_default_debug -1 all [ 3698.163331] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:49731 to 0x280000401:49825) [ 3698.163676] Lustre: lustre-OST0001: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x2c0000401:35 to 0x2c0000401:97) [ 3704.247016] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 3705.874280] Lustre: 243897:0:(mgs_llog.c:1348:mgs_modify_param()) MGS: modify general/lod.*.mdt_hash=crush (mode = 0) failed: rc = -17 [ 3713.920742] Lustre: DEBUG MARKER: == conf-sanity test 104a: Make sure user defined options are reflected in mount ========================================================== 16:18:13 (1767388693) [ 3728.869558] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 3728.873810] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 3728.879870] Lustre: Skipped 2 previous similar messages [ 3728.883339] Lustre: Skipped 1 previous similar message [ 3732.420107] LustreError: 244266:0:(obd_class.h:479:obd_check_dev()) Device 10 not setup [ 3732.423748] LustreError: 244266:0:(obd_class.h:479:obd_check_dev()) Skipped 5 previous similar messages [ 3732.460902] Lustre: server umount lustre-MDT0000 complete [ 3733.823022] LustreError: 240915:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1767388714 with bad export cookie 10318559997767292822 [ 3733.826596] LustreError: MGC192.168.203.108@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 3733.826678] LustreError: 240915:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) Skipped 6 previous similar messages [ 3733.985069] LustreError: 242132:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 3733.985973] Lustre: lustre-MDT0001-lwp-OST0000: Connection to lustre-MDT0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 3733.994620] LustreError: 242132:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 3 previous similar messages [ 3734.008138] Lustre: Skipped 2 previous similar messages [ 3739.105530] Lustre: lustre-MDT0001: Not available for connect from 0@lo (stopping) [ 3739.109279] Lustre: Skipped 5 previous similar messages [ 3748.319127] Lustre: lustre-MDT0001 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 3748.430456] Lustre: server umount lustre-MDT0001 complete [ 3756.099973] Lustre: server umount lustre-OST0000 complete [ 3772.383102] Lustre: lustre-OST0001 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 3772.475525] Lustre: server umount lustre-OST0001 complete [ 3775.377582] Lustre: DEBUG MARKER: oleg308-server.virtnet: executing set_hostid [ 3779.479972] Lustre: DEBUG MARKER: oleg308-server.virtnet: executing load_modules_local [ 3784.359141] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 3787.730539] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 3790.793472] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 3793.969220] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 3797.388110] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 3797.431738] LDISKFS-fs (dm-0): Mount option "noacl" will be removed by 3.5 [ 3797.431738] Contact linux-ldiskfs@vger.kernel.org if you think we should keep it. [ 3797.431738] [ 3797.446305] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: acl,user_xattr,errors=remount-ro,noacl,no_mbcache,nodelalloc [ 3797.568222] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 3797.590468] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 3797.636639] Lustre: lustre-MDT0000: new disk, initializing [ 3797.664451] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 3797.668097] Lustre: Skipped 3 previous similar messages [ 3797.677181] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 3799.546128] Lustre: DEBUG MARKER: oleg308-server.virtnet: executing set_default_debug -1 all [ 3805.240887] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 3805.289223] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 3805.349313] Lustre: 247057:0:(mgs_llog.c:1348:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 3805.397808] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [ 3805.401656] Lustre: Skipped 1 previous similar message [ 3805.457351] Lustre: lustre-MDT0001: new disk, initializing [ 3805.543137] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [ 3805.550113] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 3807.412312] Lustre: DEBUG MARKER: oleg308-server.virtnet: executing set_default_debug -1 all [ 3812.968073] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 3813.005040] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 3813.114711] Lustre: lustre-OST0000: new disk, initializing [ 3813.116856] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 3815.166128] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [ 3815.169219] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 3815.204550] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 3815.422076] Lustre: DEBUG MARKER: oleg308-server.virtnet: executing set_default_debug -1 all [ 3820.174373] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 3820.219489] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 3820.308594] Lustre: lustre-OST0001: new disk, initializing [ 3820.313289] Lustre: srv-lustre-OST0001: No data found on store. Initialize space. [ 3822.231137] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x00000002c0000400-0x0000000300000400]:1:ost [ 3822.236550] Lustre: cli-lustre-OST0001-super: Allocated super-sequence [0x00000002c0000400-0x0000000300000400]:1:ost] [ 3822.269393] Lustre: lustre-OST0001-osc-MDT0000: update sequence from 0x100010000 to 0x2c0000401 [ 3822.879140] Lustre: DEBUG MARKER: oleg308-server.virtnet: executing set_default_debug -1 all [ 3828.584644] Lustre: DEBUG MARKER: == conf-sanity test 104b: Mount uses last flock argument ========================================================== 16:20:07 (1767388807) [ 3833.628153] Lustre: DEBUG MARKER: == conf-sanity test 105: check file creation for ro and rw bind mnt pt ========================================================== 16:20:12 (1767388812) [ 3846.113082] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 3846.113426] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 3846.117717] LustreError: Skipped 1 previous similar message [ 3846.118624] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 3846.124971] Lustre: Skipped 1 previous similar message [ 3846.133422] Lustre: Skipped 3 previous similar messages [ 3851.505153] LustreError: 250479:0:(obd_class.h:479:obd_check_dev()) Device 19 not setup [ 3851.509111] LustreError: 250479:0:(obd_class.h:479:obd_check_dev()) Skipped 25 previous similar messages [ 3851.538547] Lustre: server umount lustre-OST0000 complete [ 3853.290513] LustreError: 250481:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-OST0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 3853.300412] LustreError: 250481:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 5 previous similar messages [ 3859.517337] Lustre: server umount lustre-MDT0000 complete [ 3861.314089] LustreError: 247050:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1767388841 with bad export cookie 10318559997767294376 [ 3861.321109] LustreError: 247050:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 3861.321625] LustreError: MGC192.168.203.108@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 3863.521524] Lustre: lustre-MDT0001-lwp-OST0001: Connection to lustre-MDT0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 3863.528808] Lustre: Skipped 3 previous similar messages [ 3867.547842] Lustre: server umount lustre-MDT0001 complete [ 3872.500301] Lustre: DEBUG MARKER: oleg308-server.virtnet: executing unload_modules_local [ 3892.191201] Lustre: lustre-OST0001 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 3892.289689] Lustre: server umount lustre-OST0001 complete [ 3895.319453] Lustre: DEBUG MARKER: oleg308-server.virtnet: executing set_hostid [ 3901.384536] Lustre: DEBUG MARKER: oleg308-server.virtnet: executing load_modules_local [ 3906.177669] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 3909.260463] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 3912.044985] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 3915.295553] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 3918.618920] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 3918.653654] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 3918.785355] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 3918.806321] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 3918.867774] Lustre: lustre-MDT0000: new disk, initializing [ 3918.911575] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 3920.700264] Lustre: DEBUG MARKER: oleg308-server.virtnet: executing set_default_debug -1 all [ 3926.195682] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 3926.234316] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 3926.290050] Lustre: 253943:0:(mgs_llog.c:1348:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 3926.419580] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 3926.421751] Lustre: Skipped 4 previous similar messages [ 3926.435611] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 3928.209893] Lustre: DEBUG MARKER: oleg308-server.virtnet: executing set_default_debug -1 all [ 3932.215369] Lustre: DEBUG MARKER: oleg308-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 3933.999675] Lustre: DEBUG MARKER: oleg308-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 3937.271166] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 3937.312111] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 3937.462530] Lustre: lustre-OST0000: new disk, initializing [ 3937.464791] Lustre: Skipped 1 previous similar message [ 3937.467740] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 3937.471453] Lustre: Skipped 2 previous similar messages [ 3938.636715] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [ 3938.641619] Lustre: Skipped 1 previous similar message [ 3938.644953] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 3938.694604] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 3940.031229] Lustre: DEBUG MARKER: oleg308-server.virtnet: executing set_default_debug -1 all [ 3944.110474] Lustre: DEBUG MARKER: oleg308-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 3949.027934] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 3949.028539] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 3949.030903] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 3949.030914] Lustre: Skipped 10 previous similar messages [ 3949.035093] Lustre: Skipped 1 previous similar message [ 3949.039470] LustreError: Skipped 1 previous similar message [ 3953.116954] Lustre: server umount lustre-OST0000 complete [ 3957.216454] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 3960.756840] Lustre: server umount lustre-MDT0000 complete [ 3962.503078] LustreError: 253934:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1767388943 with bad export cookie 10318559997767295846 [ 3962.509989] LustreError: 253934:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 3962.510349] LustreError: MGC192.168.203.108@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 3962.656217] Lustre: server umount lustre-MDT0001 complete [ 3967.587784] Lustre: DEBUG MARKER: oleg308-server.virtnet: executing unload_modules_local [ 3969.164894] Key type lgssc unregistered [ 3969.345811] LNet: 257322:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 3969.350871] LNetError: 257322:0:(acceptor.c:264:lnet_acceptor_remove_socket()) Interface ens2 not found [ 3969.360738] LNet: Removed LNI 192.168.203.108@tcp [ 3969.845208] Key type .llcrypt unregistered [ 3969.847519] Key type ._llcrypt unregistered [ 3983.102945] Key type ._llcrypt registered [ 3983.104054] Key type .llcrypt registered [ 3983.151615] Lustre: DEBUG MARKER: oleg308-server.virtnet: executing load_modules_local [ 3983.451807] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 3983.503205] alg: No test for adler32 (adler32-zlib) [ 3984.393601] Lustre: Lustre: Build Version: 2.17.0_RC4_2_g45eb11f [ 3984.499455] LNet: Added LNI 192.168.203.108@tcp [8/256/0/180] [ 3986.095162] Key type lgssc registered [ 3986.725852] Lustre: Echo OBD driver; http://www.lustre.org/ [ 3990.608176] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 3990.619656] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 3991.861116] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 3993.511793] Lustre: DEBUG MARKER: oleg308-server.virtnet: executing set_default_debug -1 all [ 3997.591592] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 3997.747774] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 3999.458102] Lustre: DEBUG MARKER: oleg308-server.virtnet: executing set_default_debug -1 all [ 4002.017841] Lustre: DEBUG MARKER: oleg308-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 4003.820700] Lustre: DEBUG MARKER: oleg308-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 4007.142668] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 4007.328921] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 4009.843770] Lustre: DEBUG MARKER: oleg308-server.virtnet: executing set_default_debug -1 all [ 4011.881230] Lustre: DEBUG MARKER: oleg308-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 4012.396703] Lustre: Setting parameter lustre-OST0000.ost.unknown_param=50 in log lustre-OST0000 [ 4013.537810] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 4013.539254] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4013.550707] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 4015.346166] LustreError: 261038:0:(obd_class.h:479:obd_check_dev()) Device 21 not setup [ 4015.397993] Lustre: server umount lustre-OST0000 complete [ 4018.144527] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 4018.146808] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4018.150552] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 4019.212393] LustreError: 261238:0:(obd_class.h:479:obd_check_dev()) Device 9 not setup [ 4019.216110] LustreError: 261238:0:(obd_class.h:479:obd_check_dev()) Skipped 3 previous similar messages [ 4019.247696] Lustre: server umount lustre-MDT0000 complete [ 4021.025635] LustreError: 258723:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1767389001 with bad export cookie 10519621147403776017 [ 4021.032455] LustreError: 258723:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) Skipped 2 previous similar messages [ 4021.033417] LustreError: MGC192.168.203.108@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 4023.181473] LustreError: 261442:0:(obd_class.h:479:obd_check_dev()) Device 16 not setup [ 4023.185221] LustreError: 261442:0:(obd_class.h:479:obd_check_dev()) Skipped 5 previous similar messages [ 4023.248950] Lustre: server umount lustre-MDT0001 complete [ 4028.226703] Lustre: DEBUG MARKER: oleg308-server.virtnet: executing unload_modules_local [ 4029.704182] Key type lgssc unregistered [ 4029.874738] LNet: 261973:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 4029.879432] LNetError: 261973:0:(acceptor.c:264:lnet_acceptor_remove_socket()) Interface ens2 not found [ 4029.889616] LNet: Removed LNI 192.168.203.108@tcp [ 4030.338175] Key type .llcrypt unregistered [ 4030.340251] Key type ._llcrypt unregistered [ 4039.232704] Key type ._llcrypt registered [ 4039.234641] Key type .llcrypt registered [ 4039.302378] Lustre: DEBUG MARKER: oleg308-server.virtnet: executing load_modules_local [ 4039.817729] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 4039.826984] alg: No test for adler32 (adler32-zlib) [ 4040.694994] Lustre: Lustre: Build Version: 2.17.0_RC4_2_g45eb11f [ 4040.792660] LNet: Added LNI 192.168.203.108@tcp [8/256/0/180] [ 4042.383193] Key type lgssc registered [ 4042.894251] Lustre: Echo OBD driver; http://www.lustre.org/ [ 4047.149767] Lustre: lustre-OST0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 4047.160797] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 4062.623328] LustreError: Failed to get MGS log lustre-sptlrpc and no local copy. [ 4067.743194] LustreError: 263123:0:(mgc_request.c:233:do_config_log_add()) MGC192.168.203.108@tcp: failed processing log, type 4: rc = -110 [ 4094.367121] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 4096.973953] Lustre: DEBUG MARKER: oleg308-server.virtnet: executing set_default_debug -1 all [ 4099.698560] Lustre: DEBUG MARKER: oleg308-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 4100.722960] LustreError: 263786:0:(obd_class.h:479:obd_check_dev()) Device 1 not setup [ 4100.784951] Lustre: server umount lustre-OST0000 complete [ 4108.486463] Lustre: DEBUG MARKER: oleg308-server.virtnet: executing unload_modules_local [ 4110.101701] Key type lgssc unregistered [ 4110.273726] LNet: 264622:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 4110.278607] LNetError: 264622:0:(acceptor.c:264:lnet_acceptor_remove_socket()) Interface ens2 not found [ 4110.287710] LNet: Removed LNI 192.168.203.108@tcp [ 4110.749698] Key type .llcrypt unregistered [ 4110.751974] Key type ._llcrypt unregistered [ 4128.923806] Key type ._llcrypt registered [ 4128.925032] Key type .llcrypt registered [ 4128.976235] Lustre: DEBUG MARKER: oleg308-server.virtnet: executing load_modules_local [ 4129.256032] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 4129.270152] alg: No test for adler32 (adler32-zlib) [ 4130.126328] Lustre: Lustre: Build Version: 2.17.0_RC4_2_g45eb11f [ 4130.227938] LNet: Added LNI 192.168.203.108@tcp [8/256/0/180] [ 4131.807182] Key type lgssc registered [ 4132.393335] Lustre: Echo OBD driver; http://www.lustre.org/ [ 4138.577682] loop0: detected capacity change from 0 to 204800000 [ 4138.669759] blk_update_request: operation not supported error, dev loop0, sector 399872 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [ 4138.673899] blk_update_request: operation not supported error, dev loop0, sector 208 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [ 4138.865228] blk_update_request: operation not supported error, dev loop0, sector 160232 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [ 4138.892958] LDISKFS-fs (loop0): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 4139.361546] loop0: detected capacity change from 0 to 204800000 [ 4139.458574] blk_update_request: operation not supported error, dev loop0, sector 399872 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [ 4139.462583] blk_update_request: operation not supported error, dev loop0, sector 208 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [ 4139.662548] blk_update_request: operation not supported error, dev loop0, sector 160232 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [ 4139.690406] LDISKFS-fs (loop0): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 4140.180624] loop0: detected capacity change from 0 to 204800000 [ 4140.297281] blk_update_request: operation not supported error, dev loop0, sector 399872 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [ 4140.304179] blk_update_request: operation not supported error, dev loop0, sector 8224 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [ 4140.368299] blk_update_request: operation not supported error, dev loop0, sector 58264 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [ 4140.436708] LDISKFS-fs (loop0): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 4140.894151] loop0: detected capacity change from 0 to 204800000 [ 4140.985850] blk_update_request: operation not supported error, dev loop0, sector 399872 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [ 4141.111309] LDISKFS-fs (loop0): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 4141.549269] loop0: detected capacity change from 0 to 204800000 [ 4141.562666] LDISKFS-fs (loop0): mounted filesystem with ordered data mode. Opts: (null) [ 4143.366811] loop0: detected capacity change from 0 to 204800000 [ 4143.385125] LDISKFS-fs (loop0): mounted filesystem with ordered data mode. Opts: (null) [ 4145.208464] loop0: detected capacity change from 0 to 204800000 [ 4145.226110] LDISKFS-fs (loop0): mounted filesystem with ordered data mode. Opts: (null) [ 4147.075285] loop0: detected capacity change from 0 to 204800000 [ 4147.095789] LDISKFS-fs (loop0): mounted filesystem with ordered data mode. Opts: (null) [ 4148.927802] loop0: detected capacity change from 0 to 204800000 [ 4148.982294] LDISKFS-fs (loop0): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 4149.024574] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 4149.031291] LDISKFS-fs (loop0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 4149.065520] Lustre: 268000:0:(osd_scrub.c:1842:osd_index_restore()) lustre-MDT0000: restore index 'fld' with [0x200000001:0x3:0x0]: rc = 0 [ 4149.070709] Lustre: 268000:0:(osd_scrub.c:1842:osd_index_restore()) lustre-MDT0000: restore index '0x2010000-MDT0000' with [0x200000005:0x3:0x0]: rc = 0 [ 4149.076595] Lustre: 268000:0:(osd_scrub.c:1842:osd_index_restore()) lustre-MDT0000: restore index '0x2010000' with [0x200000003:0xc:0x0]: rc = 0 [ 4149.083136] Lustre: 268000:0:(osd_scrub.c:1842:osd_index_restore()) lustre-MDT0000: restore index '0x1010000' with [0x200000003:0xb:0x0]: rc = 0 [ 4149.088986] Lustre: 268000:0:(osd_scrub.c:1842:osd_index_restore()) lustre-MDT0000: restore index '0x1010000-MDT0000' with [0x200000005:0x2:0x0]: rc = 0 [ 4149.095055] Lustre: 268000:0:(osd_scrub.c:1842:osd_index_restore()) lustre-MDT0000: restore index '0x10000-MDT0000' with [0x200000005:0x1:0x0]: rc = 0 [ 4149.102271] Lustre: 268000:0:(osd_scrub.c:1842:osd_index_restore()) lustre-MDT0000: restore index '0x10000' with [0x200000003:0xa:0x0]: rc = 0 [ 4149.109229] Lustre: 268000:0:(osd_scrub.c:1842:osd_index_restore()) lustre-MDT0000: restore index 'nodemap' with [0x200000003:0x3:0x0]: rc = 0 [ 4149.115633] Lustre: 268000:0:(osd_scrub.c:1842:osd_index_restore()) lustre-MDT0000: restore index '0x2010000' with [0x200000006:0x2010000:0x0]: rc = 0 [ 4149.122357] Lustre: 268000:0:(osd_scrub.c:1842:osd_index_restore()) lustre-MDT0000: restore index '0x1010000' with [0x200000006:0x1010000:0x0]: rc = 0 [ 4149.129933] Lustre: 268000:0:(osd_scrub.c:1842:osd_index_restore()) lustre-MDT0000: restore index '0x10000' with [0x200000006:0x10000:0x0]: rc = 0 [ 4149.136895] Lustre: 268000:0:(osd_scrub.c:1842:osd_index_restore()) lustre-MDT0000: restore index '0x1020000' with [0x200000006:0x1020000:0x0]: rc = 0 [ 4149.143922] Lustre: 268000:0:(osd_scrub.c:1842:osd_index_restore()) lustre-MDT0000: restore index '0x20000' with [0x200000006:0x20000:0x0]: rc = 0 [ 4149.150833] Lustre: 268000:0:(osd_scrub.c:1842:osd_index_restore()) lustre-MDT0000: restore index '0x2020000' with [0x200000006:0x2020000:0x0]: rc = 0 [ 4152.271424] LustreError: 268266:0:(obd_class.h:479:obd_check_dev()) Device 2 not setup [ 4152.340512] Lustre: server umount MGS complete [ 4152.754272] loop0: detected capacity change from 0 to 204800000 [ 4152.812894] LDISKFS-fs (loop0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 4153.008825] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 4153.028477] Lustre: lustre-MDT0000: Aborting client recovery [ 4153.435383] loop1: detected capacity change from 0 to 204800000 [ 4153.493475] LDISKFS-fs (loop1): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 4153.527317] LDISKFS-fs (loop1): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 4153.547670] Lustre: 268436:0:(osd_scrub.c:1842:osd_index_restore()) lustre-MDT0001: restore index 'fld' with [0x200000001:0x3:0x0]: rc = 0 [ 4153.551427] Lustre: 268436:0:(osd_scrub.c:1842:osd_index_restore()) lustre-MDT0001: restore index '0x2010000' with [0x200000003:0x6:0x0]: rc = 0 [ 4153.554365] Lustre: 268436:0:(osd_scrub.c:1842:osd_index_restore()) lustre-MDT0001: restore index '0x1010000-MDT0001' with [0x200000005:0x2:0x0]: rc = 0 [ 4153.558995] Lustre: 268436:0:(osd_scrub.c:1842:osd_index_restore()) lustre-MDT0001: restore index '0x1010000' with [0x200000003:0x5:0x0]: rc = 0 [ 4153.563549] Lustre: 268436:0:(osd_scrub.c:1842:osd_index_restore()) lustre-MDT0001: restore index '0x10000-MDT0001' with [0x200000005:0x1:0x0]: rc = 0 [ 4153.566393] Lustre: 268436:0:(osd_scrub.c:1842:osd_index_restore()) lustre-MDT0001: restore index '0x10000' with [0x200000003:0x4:0x0]: rc = 0 [ 4153.570754] Lustre: 268436:0:(osd_scrub.c:1842:osd_index_restore()) lustre-MDT0001: restore index '0x2010000-MDT0001' with [0x200000005:0x3:0x0]: rc = 0 [ 4153.578691] Lustre: Found index 1 for lustre-MDT0001, updating log [ 4153.659567] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 4153.694277] Lustre: lustre-MDT0001: Aborting client recovery [ 4154.098196] loop2: detected capacity change from 0 to 204800000 [ 4154.171271] LDISKFS-fs (loop2): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 4154.189716] LDISKFS-fs (loop2): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 4154.205702] Lustre: 268522:0:(osd_scrub.c:1842:osd_index_restore()) lustre-OST0000: restore index 'fld' with [0x200000001:0x3:0x0]: rc = 0 [ 4154.208639] Lustre: 268522:0:(osd_scrub.c:1842:osd_index_restore()) lustre-OST0000: restore index '0x20000' with [0x200000003:0x5:0x0]: rc = 0 [ 4154.211366] Lustre: 268522:0:(osd_scrub.c:1842:osd_index_restore()) lustre-OST0000: restore index '0x1020000-OST0000' with [0x200000005:0x2:0x0]: rc = 0 [ 4154.214182] Lustre: 268522:0:(osd_scrub.c:1842:osd_index_restore()) lustre-OST0000: restore index '0x20000-OST0000' with [0x200000005:0x1:0x0]: rc = 0 [ 4154.217049] Lustre: 268522:0:(osd_scrub.c:1842:osd_index_restore()) lustre-OST0000: restore index '0x1020000' with [0x200000003:0x6:0x0]: rc = 0 [ 4154.219766] Lustre: 268522:0:(osd_scrub.c:1842:osd_index_restore()) lustre-OST0000: restore index '0x2020000' with [0x200000003:0x7:0x0]: rc = 0 [ 4154.222472] Lustre: 268522:0:(osd_scrub.c:1842:osd_index_restore()) lustre-OST0000: restore index '0x2020000-OST0000' with [0x200000005:0x3:0x0]: rc = 0 [ 4154.280332] Lustre: Found index 0 for lustre-OST0000, updating log [ 4154.335034] Lustre: lustre-OST0000: Aborting recovery [ 4154.752774] loop3: detected capacity change from 0 to 204800000 [ 4154.811329] LDISKFS-fs (loop3): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 4154.849709] LDISKFS-fs (loop3): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 4154.866508] Lustre: 268611:0:(osd_scrub.c:1842:osd_index_restore()) lustre-OST0001: restore index 'fld' with [0x200000001:0x3:0x0]: rc = 0 [ 4154.869388] Lustre: 268611:0:(osd_scrub.c:1842:osd_index_restore()) lustre-OST0001: restore index '0x20000' with [0x200000003:0x5:0x0]: rc = 0 [ 4154.872376] Lustre: 268611:0:(osd_scrub.c:1842:osd_index_restore()) lustre-OST0001: restore index '0x2020000' with [0x200000003:0x7:0x0]: rc = 0 [ 4154.875468] Lustre: 268611:0:(osd_scrub.c:1842:osd_index_restore()) lustre-OST0001: restore index '0x20000-OST0001' with [0x200000005:0x1:0x0]: rc = 0 [ 4154.878481] Lustre: 268611:0:(osd_scrub.c:1842:osd_index_restore()) lustre-OST0001: restore index '0x1020000-OST0001' with [0x200000005:0x2:0x0]: rc = 0 [ 4154.881401] Lustre: 268611:0:(osd_scrub.c:1842:osd_index_restore()) lustre-OST0001: restore index '0x1020000' with [0x200000003:0x6:0x0]: rc = 0 [ 4154.884206] Lustre: 268611:0:(osd_scrub.c:1842:osd_index_restore()) lustre-OST0001: restore index '0x2020000-OST0001' with [0x200000005:0x3:0x0]: rc = 0 [ 4154.917240] Lustre: lustre-OST0001: Imperative Recovery not enabled, recovery window 60-180 [ 4154.919045] Lustre: Skipped 1 previous similar message [ 4154.924188] Lustre: lustre-OST0001: Aborting recovery [ 4159.983778] LustreError: 268379:0:(lod_dev.c:510:lod_sub_recovery_thread()) lustre-MDT0001-osp-MDT0000: get update log duration 7, retries 0, failed: rc = -108 [ 4163.044385] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x0:6 to 0x0:33) [ 4163.045421] Lustre: lustre-OST0001: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x0:6 to 0x0:33) [ 4163.059537] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000bd0 [ 4163.066064] Lustre: lustre-OST0001-osc-MDT0000: update sequence from 0x100010000 to 0x2c0000bd0 [ 4164.068888] Lustre: lustre-OST0000: new connection from lustre-MDT0001-mdtlov (cleaning up unused objects from 0x280000400:5 to 0x280000400:33) [ 4164.069356] Lustre: lustre-OST0001: new connection from lustre-MDT0001-mdtlov (cleaning up unused objects from 0x2c0000400:5 to 0x2c0000400:33) [ 4166.050220] LustreError: 268346:0:(mdt_identity.c:115:mdt_identity_do_upcall()) lustre-MDT0000: error invoking upcall /root/Work/Lustre/R5/lustre-release/lustre/tests/../utils/l_getidentity lustre-MDT0000 0: rc -2; check /proc/fs/lustre/mdt/lustre-MDT0000/identity_upcall, time 150us: rc = -2 [ 4179.425850] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 4179.426700] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4179.427579] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 4179.441341] Lustre: Skipped 3 previous similar messages [ 4183.437381] LustreError: 268964:0:(obd_class.h:479:obd_check_dev()) Device 10 not setup [ 4183.442023] LustreError: 268964:0:(obd_class.h:479:obd_check_dev()) Skipped 1 previous similar message [ 4183.490798] Lustre: server umount lustre-MDT0000 complete [ 4183.919178] LustreError: 268327:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1767389164 with bad export cookie 8632216007995247897 [ 4183.922338] LustreError: MGC192.168.203.108@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 4183.927432] LustreError: 268327:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) Skipped 2 previous similar messages [ 4184.543975] LustreError: 268343:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 4184.544479] Lustre: lustre-MDT0001-lwp-OST0001: Connection to lustre-MDT0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4184.545318] Lustre: lustre-MDT0001: Not available for connect from 0@lo (stopping) [ 4184.545328] Lustre: Skipped 3 previous similar messages [ 4184.558735] LustreError: 268343:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 3 previous similar messages [ 4184.567352] Lustre: Skipped 1 previous similar message [ 4188.575728] LustreError: 268343:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 4188.576084] Lustre: lustre-MDT0001: Not available for connect from 0@lo (stopping) [ 4188.590402] LustreError: 268343:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 1 previous similar message [ 4188.594029] Lustre: Skipped 2 previous similar messages [ 4193.696081] LustreError: 268912:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 4193.696117] Lustre: lustre-MDT0001: Not available for connect from 0@lo (stopping) [ 4193.705479] LustreError: 268912:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 1 previous similar message [ 4193.709052] Lustre: Skipped 1 previous similar message [ 4198.367140] Lustre: lustre-MDT0001 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 4198.398066] LustreError: 269018:0:(obd_class.h:479:obd_check_dev()) Device 18 not setup [ 4198.399649] LustreError: 269018:0:(obd_class.h:479:obd_check_dev()) Skipped 7 previous similar messages [ 4198.453165] Lustre: server umount lustre-MDT0001 complete [ 4213.215197] Lustre: lustre-OST0000 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 4213.242273] LustreError: 269070:0:(obd_class.h:479:obd_check_dev()) Device 23 not setup [ 4213.246157] LustreError: 269070:0:(obd_class.h:479:obd_check_dev()) Skipped 7 previous similar messages [ 4213.285429] Lustre: server umount lustre-OST0000 complete [ 4228.063067] Lustre: lustre-OST0001 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 4228.081724] LustreError: 269122:0:(obd_class.h:479:obd_check_dev()) Device 2 not setup [ 4228.083655] LustreError: 269122:0:(obd_class.h:479:obd_check_dev()) Skipped 3 previous similar messages [ 4228.133793] Lustre: server umount lustre-OST0001 complete [ 4231.960285] Lustre: DEBUG MARKER: == conf-sanity test 109a: test lctl clear_conf fsname ==== 16:26:51 (1767389211) [ 4244.810443] Lustre: DEBUG MARKER: oleg308-server.virtnet: executing set_hostid [ 4249.420369] Lustre: DEBUG MARKER: oleg308-server.virtnet: executing load_modules_local [ 4254.389374] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 4257.735307] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 4260.245653] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 4263.556064] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 4266.889281] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 4266.941086] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 4267.072712] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 4267.091840] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 4267.149206] Lustre: lustre-MDT0000: new disk, initializing [ 4267.190726] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 4267.203475] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 4269.071909] Lustre: DEBUG MARKER: oleg308-server.virtnet: executing set_default_debug -1 all [ 4274.793085] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 4274.834893] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 4274.886739] Lustre: 272667:0:(mgs_llog.c:1348:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 4274.932904] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [ 4274.936439] Lustre: Skipped 1 previous similar message [ 4274.986861] Lustre: lustre-MDT0001: new disk, initializing [ 4275.030782] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 4275.045375] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [ 4275.049511] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 4276.832407] Lustre: DEBUG MARKER: oleg308-server.virtnet: executing set_default_debug -1 all [ 4280.987445] Lustre: DEBUG MARKER: oleg308-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 4282.861188] Lustre: DEBUG MARKER: oleg308-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 4286.293054] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 4286.336201] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 4286.511916] Lustre: lustre-OST0000: new disk, initializing [ 4286.514820] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 4286.560586] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 4289.206527] Lustre: DEBUG MARKER: oleg308-server.virtnet: executing set_default_debug -1 all [ 4293.405653] Lustre: DEBUG MARKER: oleg308-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 4294.155932] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [ 4294.162553] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 4294.225214] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 4301.525896] Lustre: Setting parameter lustre-MDT0000.mdd.atime_diff=62 in log lustre-MDT0000 [ 4310.320353] Lustre: Modifying parameter lustre-MDT0000.mdd.atime_diff=63 in log lustre-MDT0000 [ 4318.366370] Lustre: Setting parameter lustre-client.llite.max_read_ahead_mb=32 in log lustre-client [ 4325.177841] Lustre: Modifying parameter lustre-client.llite.max_read_ahead_mb=64 in log lustre-client [ 4360.676116] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 4360.679381] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4360.681886] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 4360.688263] Lustre: Skipped 1 previous similar message [ 4360.694351] Lustre: Skipped 1 previous similar message [ 4366.323141] LustreError: 276384:0:(obd_class.h:479:obd_check_dev()) Device 19 not setup [ 4366.327070] LustreError: 276384:0:(obd_class.h:479:obd_check_dev()) Skipped 5 previous similar messages [ 4366.392660] Lustre: server umount lustre-OST0000 complete [ 4370.914818] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4370.923703] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 4370.927404] Lustre: Skipped 3 previous similar messages [ 4374.400461] Lustre: server umount lustre-MDT0000 complete [ 4376.034485] LustreError: 272678:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 4376.043909] LustreError: 272678:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 1 previous similar message [ 4376.234995] LustreError: 272658:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1767389356 with bad export cookie 8632216007995250606 [ 4376.241761] LustreError: 272658:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) Skipped 2 previous similar messages [ 4376.244493] LustreError: MGC192.168.203.108@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 4376.415394] Lustre: server umount lustre-MDT0001 complete [ 4380.029565] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 4381.476217] Lustre: DEBUG MARKER: oleg308-server.virtnet: executing set_default_debug -1 all [ 4383.640854] LustreError: 277729:0:(obd_class.h:479:obd_check_dev()) Device 2 not setup [ 4383.644712] LustreError: 277729:0:(obd_class.h:479:obd_check_dev()) Skipped 17 previous similar messages [ 4383.677903] Lustre: server umount MGS complete [ 4388.939314] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 4389.168684] LustreError: 278351:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 4389.180084] LustreError: 278363:0:(qmt_pool.c:1523:qmt_pool_add_rem()) lustre-QMT0000: can't remove lustre-OST0000_UUID pool 'pool1': rc = -22 [ 4389.219371] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 4391.007891] Lustre: DEBUG MARKER: oleg308-server.virtnet: executing set_default_debug -1 all [ 4394.827570] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 4396.737206] Lustre: DEBUG MARKER: oleg308-server.virtnet: executing set_default_debug -1 all [ 4399.282371] Lustre: DEBUG MARKER: oleg308-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 4401.011973] Lustre: DEBUG MARKER: oleg308-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 4404.301394] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 4406.995361] Lustre: DEBUG MARKER: oleg308-server.virtnet: executing set_default_debug -1 all [ 4409.606403] Lustre: DEBUG MARKER: oleg308-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 4436.448287] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 4436.453868] LustreError: Skipped 1 previous similar message [ 4436.457024] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4436.465554] Lustre: Skipped 1 previous similar message [ 4436.469276] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 4439.526167] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 4442.034113] LustreError: 281011:0:(obd_class.h:479:obd_check_dev()) Device 21 not setup [ 4442.038802] LustreError: 281011:0:(obd_class.h:479:obd_check_dev()) Skipped 1 previous similar message [ 4442.094755] Lustre: server umount lustre-OST0000 complete [ 4444.642142] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4444.649380] Lustre: Skipped 2 previous similar messages [ 4450.035153] Lustre: server umount lustre-MDT0000 complete [ 4451.804511] LustreError: 279972:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1767389432 with bad export cookie 8632216007995252517 [ 4451.810172] LustreError: MGC192.168.203.108@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 4451.814306] LustreError: 279972:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 4451.948044] Lustre: server umount lustre-MDT0001 complete [ 4456.987904] Lustre: DEBUG MARKER: oleg308-server.virtnet: executing unload_modules_local [ 4458.461477] Key type lgssc unregistered [ 4458.617679] LNet: 281947:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 4458.622814] LNetError: 281947:0:(acceptor.c:264:lnet_acceptor_remove_socket()) Interface ens2 not found [ 4458.634580] LNet: Removed LNI 192.168.203.108@tcp [ 4459.103919] Key type .llcrypt unregistered [ 4459.106126] Key type ._llcrypt unregistered [ 4475.545461] Key type ._llcrypt registered [ 4475.547284] Key type .llcrypt registered [ 4475.609787] Lustre: DEBUG MARKER: oleg308-server.virtnet: executing set_hostid [ 4483.340442] Lustre: DEBUG MARKER: oleg308-server.virtnet: executing load_modules_local [ 4483.634621] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 4483.640502] alg: No test for adler32 (adler32-zlib) [ 4484.551909] Lustre: Lustre: Build Version: 2.17.0_RC4_2_g45eb11f [ 4484.665374] LNet: Added LNI 192.168.203.108@tcp [8/256/0/180] [ 4486.287171] Key type lgssc registered [ 4486.843005] Lustre: Echo OBD driver; http://www.lustre.org/ [ 4491.165595] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 4494.529672] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 4497.630285] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 4500.829333] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 4504.228290] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 4504.259770] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 4504.272711] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 4505.365525] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 4505.379186] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 4505.426730] Lustre: lustre-MDT0000: new disk, initializing [ 4505.451947] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 4505.459653] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 4507.159903] Lustre: DEBUG MARKER: oleg308-server.virtnet: executing set_default_debug -1 all [ 4512.549293] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 4512.590660] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 4512.643655] Lustre: 285854:0:(mgs_llog.c:1348:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 4512.680816] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [ 4512.683522] Lustre: Skipped 1 previous similar message [ 4512.737081] Lustre: lustre-MDT0001: new disk, initializing [ 4512.785615] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 4512.800494] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [ 4512.804684] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 4514.162801] Lustre: DEBUG MARKER: oleg308-server.virtnet: executing set_default_debug -1 all [ 4517.957689] Lustre: DEBUG MARKER: oleg308-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 4519.583340] Lustre: DEBUG MARKER: oleg308-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 4522.764595] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 4522.803171] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 4522.947742] Lustre: lustre-OST0000: new disk, initializing [ 4522.950702] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 4522.985122] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 4525.339240] Lustre: DEBUG MARKER: oleg308-server.virtnet: executing set_default_debug -1 all [ 4529.240233] Lustre: DEBUG MARKER: oleg308-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 4531.208496] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [ 4531.214594] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 4531.269761] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 4538.413525] Lustre: Setting parameter lustre-MDT0000.mdd.atime_diff=62 in log lustre-MDT0000 [ 4542.941546] Lustre: Modifying parameter lustre-MDT0000.mdd.atime_diff=63 in log lustre-MDT0000 [ 4552.384228] Lustre: Setting parameter lustre-client.llite.max_read_ahead_mb=32 in log lustre-client [ 4558.158468] Lustre: Modifying parameter lustre-client.llite.max_read_ahead_mb=64 in log lustre-client [ 4597.732125] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4597.733224] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 4597.741313] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 4597.745433] LustreError: Skipped 1 previous similar message [ 4597.754371] Lustre: Skipped 1 previous similar message [ 4599.154647] LustreError: 289465:0:(obd_class.h:479:obd_check_dev()) Device 19 not setup [ 4599.213437] Lustre: server umount lustre-OST0000 complete [ 4602.850185] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4602.857428] Lustre: Skipped 1 previous similar message [ 4602.861133] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 4602.864859] Lustre: Skipped 1 previous similar message [ 4607.185533] LustreError: 289667:0:(obd_class.h:479:obd_check_dev()) Device 22 not setup [ 4607.189446] LustreError: 289667:0:(obd_class.h:479:obd_check_dev()) Skipped 3 previous similar messages [ 4607.228849] Lustre: server umount lustre-MDT0000 complete [ 4609.019890] LustreError: 285844:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1767389589 with bad export cookie 11608178369631690929 [ 4609.025192] LustreError: MGC192.168.203.108@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 4609.026631] LustreError: 285844:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) Skipped 2 previous similar messages [ 4609.126220] LustreError: 289868:0:(obd_class.h:479:obd_check_dev()) Device 21 not setup [ 4609.130576] LustreError: 289868:0:(obd_class.h:479:obd_check_dev()) Skipped 5 previous similar messages [ 4609.192575] Lustre: server umount lustre-MDT0001 complete [ 4613.305665] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 4615.106851] Lustre: DEBUG MARKER: oleg308-server.virtnet: executing set_default_debug -1 all [ 4617.383742] LustreError: 290808:0:(obd_class.h:479:obd_check_dev()) Device 2 not setup [ 4617.388613] LustreError: 290808:0:(obd_class.h:479:obd_check_dev()) Skipped 7 previous similar messages [ 4617.424866] Lustre: server umount MGS complete [ 4622.371427] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 4622.590963] LustreError: 291431:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 4622.602340] LustreError: 291443:0:(qmt_pool.c:1523:qmt_pool_add_rem()) lustre-QMT0000: can't remove lustre-OST0000_UUID pool 'pool1': rc = -22 [ 4622.603139] LustreError: 291431:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 3 previous similar messages [ 4622.644128] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 4624.459022] Lustre: DEBUG MARKER: oleg308-server.virtnet: executing set_default_debug -1 all [ 4627.938169] LustreError: 291431:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 4628.422569] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 4628.599300] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 4630.291559] Lustre: DEBUG MARKER: oleg308-server.virtnet: executing set_default_debug -1 all [ 4632.885332] Lustre: DEBUG MARKER: oleg308-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 4634.594279] Lustre: DEBUG MARKER: oleg308-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 4637.787164] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 4637.971223] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 4640.260653] Lustre: DEBUG MARKER: oleg308-server.virtnet: executing set_default_debug -1 all [ 4642.754585] Lustre: DEBUG MARKER: oleg308-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 4669.920239] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 4669.924300] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4669.929989] Lustre: Skipped 1 previous similar message [ 4669.932623] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 4674.019764] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 4674.023905] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4674.023992] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 4675.378183] LustreError: 294088:0:(obd_class.h:479:obd_check_dev()) Device 21 not setup [ 4675.382073] LustreError: 294088:0:(obd_class.h:479:obd_check_dev()) Skipped 1 previous similar message [ 4675.435342] Lustre: server umount lustre-OST0000 complete [ 4679.138002] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4679.146456] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 4679.150186] Lustre: Skipped 2 previous similar messages [ 4683.373064] Lustre: server umount lustre-MDT0000 complete [ 4684.255886] LustreError: 291430:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 4684.264684] LustreError: 291430:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 1 previous similar message [ 4685.023980] LustreError: 291456:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1767389665 with bad export cookie 11608178369631692770 [ 4685.028443] LustreError: MGC192.168.203.108@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 4685.030516] LustreError: 291456:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) Skipped 2 previous similar messages [ 4685.128175] LustreError: 294492:0:(obd_class.h:479:obd_check_dev()) Device 16 not setup [ 4685.130272] LustreError: 294492:0:(obd_class.h:479:obd_check_dev()) Skipped 9 previous similar messages [ 4685.182051] Lustre: server umount lustre-MDT0001 complete [ 4689.840309] Lustre: DEBUG MARKER: oleg308-server.virtnet: executing unload_modules_local [ 4691.313524] Key type lgssc unregistered [ 4691.452549] LNet: 295023:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 4691.456204] LNetError: 295023:0:(acceptor.c:264:lnet_acceptor_remove_socket()) Interface ens2 not found [ 4691.463565] LNet: Removed LNI 192.168.203.108@tcp [ 4691.855668] Key type .llcrypt unregistered [ 4691.857884] Key type ._llcrypt unregistered [ 4700.837533] Key type ._llcrypt registered [ 4700.839378] Key type .llcrypt registered [ 4700.899944] Lustre: DEBUG MARKER: oleg308-server.virtnet: executing set_hostid [ 4707.793703] Lustre: DEBUG MARKER: oleg308-server.virtnet: executing load_modules_local [ 4708.089725] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 4708.116277] alg: No test for adler32 (adler32-zlib) [ 4708.967950] Lustre: Lustre: Build Version: 2.17.0_RC4_2_g45eb11f [ 4709.071099] LNet: Added LNI 192.168.203.108@tcp [8/256/0/180] [ 4710.647155] Key type lgssc registered [ 4711.214790] Lustre: Echo OBD driver; http://www.lustre.org/ [ 4715.254130] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 4718.130236] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 4722.013876] Lustre: DEBUG MARKER: oleg308-server.virtnet: executing load_modules_local [ 4725.388741] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 4728.558822] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 4728.591764] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 4728.600132] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 4729.712265] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 4729.728868] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 4729.780021] Lustre: lustre-MDT0000: new disk, initializing [ 4729.818577] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 4729.831380] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 4731.677847] Lustre: DEBUG MARKER: oleg308-server.virtnet: executing set_default_debug -1 all [ 4737.182632] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 4737.217704] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 4737.333394] Lustre: lustre-OST0000: new disk, initializing [ 4737.336757] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 4737.339446] Lustre: Skipped 1 previous similar message [ 4737.367851] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 4739.974267] Lustre: DEBUG MARKER: oleg308-server.virtnet: executing set_default_debug -1 all [ 4744.724828] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:0:ost [ 4744.731656] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:0:ost] [ 4744.746076] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x240000400 [ 4995.553579] Lustre: lustre-MDT0000-lwp-OST0000: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4995.563049] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 4999.712246] LustreError: 299951:0:(obd_class.h:479:obd_check_dev()) Device 13 not setup [ 4999.836614] Lustre: server umount lustre-MDT0000 complete [ 5001.381688] LustreError: 298188:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1767389982 with bad export cookie 5986498510840467177 [ 5001.385369] LustreError: MGC192.168.203.108@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 5001.387527] LustreError: 298188:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) Skipped 2 previous similar messages [ 5001.414060] LustreError: 300153:0:(obd_class.h:479:obd_check_dev()) Device 2 not setup [ 5001.416469] LustreError: 300153:0:(obd_class.h:479:obd_check_dev()) Skipped 3 previous similar messages [ 5001.468276] Lustre: server umount lustre-OST0000 complete [ 5012.597624] Lustre: DEBUG MARKER: oleg308-server.virtnet: executing set_hostid [ 5016.329973] Lustre: DEBUG MARKER: oleg308-server.virtnet: executing load_modules_local [ 5021.131131] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 5024.426124] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 5027.563932] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 5030.883759] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 5034.263098] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 5034.304843] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 5034.433092] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 5034.453652] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 5034.510358] Lustre: lustre-MDT0000: new disk, initializing [ 5034.549023] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 5034.561300] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 5036.433329] Lustre: DEBUG MARKER: oleg308-server.virtnet: executing set_default_debug -1 all [ 5042.094431] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 5042.137586] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 5042.194941] Lustre: 303192:0:(mgs_llog.c:1348:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 5042.240370] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [ 5042.243884] Lustre: Skipped 1 previous similar message [ 5042.296635] Lustre: lustre-MDT0001: new disk, initializing [ 5042.339693] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 5042.356532] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [ 5042.360303] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 5044.018403] Lustre: DEBUG MARKER: oleg308-server.virtnet: executing set_default_debug -1 all [ 5048.137314] Lustre: DEBUG MARKER: oleg308-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 5049.844653] Lustre: DEBUG MARKER: oleg308-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 5053.157729] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 5053.198595] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 5053.346565] Lustre: lustre-OST0000: new disk, initializing [ 5053.350330] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 5053.404599] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 5053.934891] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [ 5053.939273] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 5053.991179] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 5056.074925] Lustre: DEBUG MARKER: oleg308-server.virtnet: executing set_default_debug -1 all [ 5059.818288] Lustre: DEBUG MARKER: oleg308-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 5061.495425] Lustre: DEBUG MARKER: oleg308-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 [ 5061.592656] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [ 5063.294621] Lustre: DEBUG MARKER: oleg308-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid 50 [ 5063.404317] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [ 5069.285582] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 5069.287642] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5069.289373] LustreError: Skipped 1 previous similar message [ 5069.290052] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 5069.299354] Lustre: Skipped 1 previous similar message [ 5070.322996] LustreError: 306133:0:(obd_class.h:479:obd_check_dev()) Device 19 not setup [ 5070.325551] LustreError: 306133:0:(obd_class.h:479:obd_check_dev()) Skipped 3 previous similar messages [ 5070.383728] Lustre: server umount lustre-OST0000 complete [ 5073.376421] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 5073.379761] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5073.385854] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 5073.390342] Lustre: Skipped 1 previous similar message [ 5078.160540] LustreError: 306334:0:(obd_class.h:479:obd_check_dev()) Device 22 not setup [ 5078.165133] LustreError: 306334:0:(obd_class.h:479:obd_check_dev()) Skipped 3 previous similar messages [ 5078.210610] Lustre: server umount lustre-MDT0000 complete [ 5079.841905] LustreError: 303182:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1767390060 with bad export cookie 5986498510843407968 [ 5079.847271] LustreError: MGC192.168.203.108@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 5079.849301] LustreError: 303182:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 5080.017741] Lustre: server umount lustre-MDT0001 complete [ 5081.477608] Lustre: DEBUG MARKER: SKIP: conf-sanity test_111 skipping SLOW test 111 [ 5082.194690] Lustre: DEBUG MARKER: == conf-sanity test 112a: mount OST with no_create option ========================================================== 16:41:01 (1767390061) [ 5085.406261] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 5085.663990] LustreError: 307107:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 5085.678272] LustreError: 307107:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 3 previous similar messages [ 5085.709161] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 5087.466314] Lustre: DEBUG MARKER: oleg308-server.virtnet: executing set_default_debug -1 all [ 5090.786117] LustreError: 307107:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 5091.669067] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 5093.826832] Lustre: DEBUG MARKER: oleg308-server.virtnet: executing set_default_debug -1 all [ 5096.390668] Lustre: DEBUG MARKER: oleg308-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 5098.062460] Lustre: DEBUG MARKER: oleg308-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 5101.382435] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 5104.132417] Lustre: DEBUG MARKER: oleg308-server.virtnet: executing set_default_debug -1 all [ 5106.796968] Lustre: DEBUG MARKER: oleg308-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 5109.878519] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 5109.907268] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 5109.989078] Lustre: lustre-OST0001: new disk, initializing [ 5109.992516] Lustre: srv-lustre-OST0001: No data found on store. Initialize space. [ 5110.040553] Lustre: lustre-OST0001: Imperative Recovery not enabled, recovery window 60-180 [ 5110.044605] Lustre: Skipped 2 previous similar messages [ 5110.814361] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x00000002c0000400-0x0000000300000400]:1:ost [ 5110.821495] Lustre: cli-lustre-OST0001-super: Allocated super-sequence [0x00000002c0000400-0x0000000300000400]:1:ost] [ 5110.843249] LustreError: 309701:0:(osp_precreate.c:969:osp_precreate_cleanup_orphans()) lustre-OST0001-osc-MDT0001: cannot cleanup orphans: rc = -1 [ 5111.926895] LustreError: 309712:0:(osp_precreate.c:969:osp_precreate_cleanup_orphans()) lustre-OST0001-osc-MDT0000: cannot cleanup orphans: rc = -1 [ 5111.933170] LustreError: 309712:0:(osp_precreate.c:969:osp_precreate_cleanup_orphans()) Skipped 2 previous similar messages [ 5112.603798] Lustre: DEBUG MARKER: oleg308-server.virtnet: executing set_default_debug -1 all [ 5112.998153] LustreError: 309701:0:(osp_precreate.c:969:osp_precreate_cleanup_orphans()) lustre-OST0001-osc-MDT0001: cannot cleanup orphans: rc = -1 [ 5113.005809] LustreError: 309701:0:(osp_precreate.c:969:osp_precreate_cleanup_orphans()) Skipped 1 previous similar message [ 5115.046698] LustreError: 309701:0:(osp_precreate.c:969:osp_precreate_cleanup_orphans()) lustre-OST0001-osc-MDT0001: cannot cleanup orphans: rc = -1 [ 5115.053291] LustreError: 309701:0:(osp_precreate.c:969:osp_precreate_cleanup_orphans()) Skipped 3 previous similar messages [ 5117.313094] Lustre: DEBUG MARKER: oleg308-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 [ 5117.416838] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [ 5119.139089] LustreError: 309712:0:(osp_precreate.c:969:osp_precreate_cleanup_orphans()) lustre-OST0001-osc-MDT0000: cannot cleanup orphans: rc = -1 [ 5119.141673] LustreError: 309712:0:(osp_precreate.c:969:osp_precreate_cleanup_orphans()) Skipped 7 previous similar messages [ 5119.145634] Lustre: DEBUG MARKER: oleg308-client.virtnet: executing wait_import_state (FULL|IDLE) osc.lustre-OST0000-osc-ffff9d4548d26000.ost_server_uuid 50 [ 5119.906170] Lustre: DEBUG MARKER: osc.lustre-OST0000-osc-ffff9d4548d26000.ost_server_uuid in FULL state after 0 sec [ 5121.276465] Lustre: DEBUG MARKER: oleg308-client.virtnet: executing wait_import_state (FULL|IDLE) osc.lustre-OST0001-osc-ffff9d4548d26000.ost_server_uuid 50 [ 5122.046295] Lustre: DEBUG MARKER: osc.lustre-OST0001-osc-ffff9d4548d26000.ost_server_uuid in FULL state after 0 sec [ 5123.248367] Lustre: lustre-OST0001-osc-MDT0000: update sequence from 0x100010000 to 0x2c0000401 [ 5136.325076] Lustre: lustre-OST0001: Not available for connect from 192.168.203.8@tcp (stopping) [ 5136.329144] Lustre: Skipped 2 previous similar messages [ 5137.898636] Lustre: lustre-OST0001-osc-MDT0001: Connection to lustre-OST0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5137.907089] Lustre: Skipped 2 previous similar messages [ 5140.337032] LustreError: 310913:0:(obd_class.h:479:obd_check_dev()) Device 25 not setup [ 5140.340834] LustreError: 310913:0:(obd_class.h:479:obd_check_dev()) Skipped 13 previous similar messages [ 5140.368817] Lustre: server umount lustre-OST0001 complete [ 5141.431512] LustreError: 309829:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-OST0001: not available for connect from 192.168.203.8@tcp (no target). If you are running an HA pair check that the target is mounted on the other server. [ 5143.013817] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5143.014300] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 5143.016618] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 5143.016636] Lustre: Skipped 2 previous similar messages [ 5143.021139] Lustre: Skipped 1 previous similar message [ 5148.135976] LustreError: 308705:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 5148.145308] LustreError: 308705:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 3 previous similar messages [ 5148.721135] LustreError: 311116:0:(obd_class.h:479:obd_check_dev()) Device 21 not setup [ 5148.724898] LustreError: 311116:0:(obd_class.h:479:obd_check_dev()) Skipped 3 previous similar messages [ 5148.776585] Lustre: server umount lustre-OST0000 complete [ 5153.250829] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 5153.251539] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5153.264987] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 5153.268685] Lustre: Skipped 4 previous similar messages [ 5156.757489] Lustre: server umount lustre-MDT0000 complete [ 5158.368780] LustreError: 307106:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 5158.378770] LustreError: 307106:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 1 previous similar message [ 5158.474994] LustreError: 307088:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1767390139 with bad export cookie 5986498510843408850 [ 5158.481653] LustreError: MGC192.168.203.108@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 5158.483173] LustreError: 307088:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 5158.641352] Lustre: server umount lustre-MDT0001 complete [ 5163.328515] Lustre: DEBUG MARKER: oleg308-server.virtnet: executing unload_modules_local [ 5164.652482] Key type lgssc unregistered [ 5164.779390] LNet: 312052:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 5164.781323] LNetError: 312052:0:(acceptor.c:264:lnet_acceptor_remove_socket()) Interface ens2 not found [ 5164.788492] LNet: Removed LNI 192.168.203.108@tcp [ 5165.131167] Key type .llcrypt unregistered [ 5165.132117] Key type ._llcrypt unregistered [ 5177.352306] Key type ._llcrypt registered [ 5177.353728] Key type .llcrypt registered [ 5177.420369] Lustre: DEBUG MARKER: oleg308-server.virtnet: executing load_modules_local [ 5177.905664] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 5177.925220] alg: No test for adler32 (adler32-zlib) [ 5178.845952] Lustre: Lustre: Build Version: 2.17.0_RC4_2_g45eb11f [ 5178.977244] LNet: Added LNI 192.168.203.108@tcp [8/256/0/180] [ 5180.591153] Key type lgssc registered [ 5181.263487] Lustre: Echo OBD driver; http://www.lustre.org/ [ 5185.344419] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 5185.355929] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 5186.588788] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 5188.382034] Lustre: DEBUG MARKER: oleg308-server.virtnet: executing set_default_debug -1 all [ 5190.712971] Lustre: DEBUG MARKER: oleg308-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 5193.754361] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 5193.914185] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 5195.850225] Lustre: DEBUG MARKER: oleg308-server.virtnet: executing set_default_debug -1 all [ 5200.541317] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 5200.708915] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 5202.967696] Lustre: DEBUG MARKER: oleg308-server.virtnet: executing set_default_debug -1 all [ 5203.823481] LustreError: 315423:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 5203.833922] LustreError: 315423:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 2 previous similar messages [ 5205.484835] Lustre: DEBUG MARKER: oleg308-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 5208.860806] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 5208.985590] Lustre: lustre-OST0001: Imperative Recovery not enabled, recovery window 60-180 [ 5211.576381] Lustre: DEBUG MARKER: oleg308-server.virtnet: executing set_default_debug -1 all [ 5213.119517] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:36 to 0x280000401:65) [ 5213.121034] Lustre: lustre-OST0001: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x2c0000401:3 to 0x2c0000401:33) [ 5214.171805] Lustre: DEBUG MARKER: oleg308-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0001-osc-[-0-9a-f]*.ost_server_uuid [ 5217.360415] Lustre: DEBUG MARKER: oleg308-server.virtnet: executing wait_import_state (FULL|IDLE) os[cp].lustre-OST0001-osc-MDT0001.ost_server_uuid 50 [ 5217.494631] Lustre: DEBUG MARKER: os[cp].lustre-OST0001-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [ 5300.196321] Lustre: lustre-OST0001-osc-MDT0001: Connection to lustre-OST0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5300.202266] LustreError: lustre-OST0001-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 5300.204775] Lustre: lustre-OST0001: Not available for connect from 0@lo (stopping) [ 5300.213019] Lustre: Skipped 1 previous similar message [ 5305.314800] Lustre: lustre-OST0001: Not available for connect from 0@lo (stopping) [ 5305.522559] LustreError: 316853:0:(obd_class.h:479:obd_check_dev()) Device 27 not setup [ 5305.548777] Lustre: server umount lustre-OST0001 complete [ 5310.436118] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 5310.438583] LustreError: 316359:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 5310.440190] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5310.440204] Lustre: Skipped 1 previous similar message [ 5310.442158] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 5310.450799] LustreError: 316359:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 1 previous similar message [ 5310.458032] Lustre: Skipped 2 previous similar messages [ 5315.553987] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 5315.554145] LustreError: 314984:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 5315.557796] Lustre: Skipped 1 previous similar message [ 5315.570559] LustreError: 314984:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 1 previous similar message [ 5320.674321] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 5320.674439] LustreError: 315423:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 5320.680660] Lustre: Skipped 1 previous similar message [ 5320.689768] LustreError: 315423:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 1 previous similar message [ 5322.207137] Lustre: lustre-OST0000 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 5322.240145] LustreError: 317054:0:(obd_class.h:479:obd_check_dev()) Device 23 not setup [ 5322.244029] LustreError: 317054:0:(obd_class.h:479:obd_check_dev()) Skipped 3 previous similar messages [ 5322.299565] Lustre: server umount lustre-OST0000 complete [ 5325.794221] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5325.801321] Lustre: Skipped 2 previous similar messages [ 5330.191069] LustreError: 317259:0:(obd_class.h:479:obd_check_dev()) Device 10 not setup [ 5330.195138] LustreError: 317259:0:(obd_class.h:479:obd_check_dev()) Skipped 3 previous similar messages [ 5330.224915] Lustre: server umount lustre-MDT0000 complete [ 5330.912599] LustreError: 314241:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 5330.923169] LustreError: 314241:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 1 previous similar message [ 5331.944799] LustreError: 313411:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1767390312 with bad export cookie 15897142589732072441 [ 5331.950751] LustreError: MGC192.168.203.108@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 5331.953046] LustreError: 313411:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 5332.096610] Lustre: server umount lustre-MDT0001 complete [ 5336.720489] Lustre: DEBUG MARKER: oleg308-server.virtnet: executing unload_modules_local [ 5338.243733] Key type lgssc unregistered [ 5338.396850] LNet: 317993:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 5338.401941] LNetError: 317993:0:(acceptor.c:264:lnet_acceptor_remove_socket()) Interface ens2 not found [ 5338.413701] LNet: Removed LNI 192.168.203.108@tcp [ 5338.897974] Key type .llcrypt unregistered [ 5338.900103] Key type ._llcrypt unregistered [ 5362.044711] Key type ._llcrypt registered [ 5362.048239] Key type .llcrypt registered [ 5362.117732] Lustre: DEBUG MARKER: oleg308-server.virtnet: executing load_modules_local [ 5362.536965] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 5362.567301] alg: No test for adler32 (adler32-zlib) [ 5363.450776] Lustre: Lustre: Build Version: 2.17.0_RC4_2_g45eb11f [ 5363.568576] LNet: Added LNI 192.168.203.108@tcp [8/256/0/180] [ 5365.175151] Key type lgssc registered [ 5365.817703] Lustre: Echo OBD driver; http://www.lustre.org/ [ 5370.349069] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 5370.360689] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 5371.591741] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 5373.472604] Lustre: DEBUG MARKER: oleg308-server.virtnet: executing set_default_debug -1 all [ 5377.629759] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 5377.798075] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 5379.701602] Lustre: DEBUG MARKER: oleg308-server.virtnet: executing set_default_debug -1 all [ 5381.124232] Lustre: Modifying parameter general.debug_raw_pointers=Y in log params [ 5384.036351] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 5384.253376] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 5386.969927] Lustre: DEBUG MARKER: oleg308-server.virtnet: executing set_default_debug -1 all [ 5387.369578] LustreError: 321714:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 5387.380653] LustreError: 321714:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 2 previous similar messages [ 5391.056670] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 5391.179624] Lustre: lustre-OST0001: Imperative Recovery not enabled, recovery window 60-180 [ 5393.775259] Lustre: DEBUG MARKER: oleg308-server.virtnet: executing set_default_debug -1 all [ 5396.349930] Lustre: lustre-OST0001: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x2c0000401:3 to 0x2c0000401:65) [ 5396.352397] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:36 to 0x280000401:97) [ 5402.973872] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 5411.532043] Lustre: Setting parameter general.lod.*.mdt_hash=crush in log params [ 5422.051430] Lustre: lustre-MDT0000-lwp-OST0000: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5422.054649] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 5422.058361] Lustre: Skipped 2 previous similar messages [ 5422.060797] Lustre: Skipped 1 previous similar message [ 5427.172052] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 5427.175878] Lustre: Skipped 4 previous similar messages [ 5427.348916] LustreError: 323665:0:(obd_class.h:479:obd_check_dev()) Device 10 not setup [ 5427.384279] Lustre: server umount lustre-MDT0000 complete [ 5429.128335] LustreError: 320235:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1767390409 with bad export cookie 9335818997705791556 [ 5429.135110] LustreError: MGC192.168.203.108@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 5429.136050] LustreError: 320235:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) Skipped 3 previous similar messages [ 5429.235840] LustreError: 323868:0:(obd_class.h:479:obd_check_dev()) Device 18 not setup [ 5429.240095] LustreError: 323868:0:(obd_class.h:479:obd_check_dev()) Skipped 7 previous similar messages [ 5429.304928] Lustre: server umount lustre-MDT0001 complete [ 5440.507188] LustreError: 324069:0:(obd_class.h:479:obd_check_dev()) Device 23 not setup [ 5440.513125] LustreError: 324069:0:(obd_class.h:479:obd_check_dev()) Skipped 7 previous similar messages [ 5440.540771] Lustre: server umount lustre-OST0000 complete [ 5452.601089] LustreError: 324270:0:(obd_class.h:479:obd_check_dev()) Device 2 not setup [ 5452.606925] LustreError: 324270:0:(obd_class.h:479:obd_check_dev()) Skipped 3 previous similar messages [ 5452.679152] Lustre: server umount lustre-OST0001 complete [ 5460.607815] Lustre: DEBUG MARKER: oleg308-server.virtnet: executing load_modules_local [ 5465.569734] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 5465.761144] LustreError: 325449:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 5465.788807] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 5467.636217] Lustre: DEBUG MARKER: oleg308-server.virtnet: executing set_default_debug -1 all [ 5471.201120] LustreError: 325449:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 5471.848597] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 5473.764286] Lustre: DEBUG MARKER: oleg308-server.virtnet: executing set_default_debug -1 all [ 5475.227906] Lustre: 326558:0:(mgs_llog.c:1348:mgs_modify_param()) MGS: modify general/debug_raw_pointers=Y (mode = 0) failed: rc = -17 [ 5478.478675] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 5478.654382] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 5478.658014] Lustre: Skipped 1 previous similar message [ 5481.271886] Lustre: DEBUG MARKER: oleg308-server.virtnet: executing set_default_debug -1 all [ 5481.775573] LustreError: 326914:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 5485.624415] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 5488.498586] Lustre: DEBUG MARKER: oleg308-server.virtnet: executing set_default_debug -1 all [ 5490.555223] Lustre: lustre-OST0001: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x2c0000401:3 to 0x2c0000401:97) [ 5490.557069] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:36 to 0x280000401:129) [ 5497.875320] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 5499.705163] Lustre: 328406:0:(mgs_llog.c:1348:mgs_modify_param()) MGS: modify general/lod.*.mdt_hash=crush (mode = 0) failed: rc = -17 [ 5506.029050] Lustre: lustre-MDT0000-lwp-OST0000: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5506.034918] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 5506.036903] Lustre: Skipped 2 previous similar messages [ 5508.065188] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 5508.073057] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5508.085340] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 5508.088911] Lustre: Skipped 1 previous similar message [ 5511.145772] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5512.211584] LustreError: 328510:0:(obd_class.h:479:obd_check_dev()) Device 10 not setup [ 5512.215460] LustreError: 328510:0:(obd_class.h:479:obd_check_dev()) Skipped 5 previous similar messages [ 5512.248864] Lustre: server umount lustre-MDT0000 complete [ 5513.997958] LustreError: 325430:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1767390494 with bad export cookie 9335818997705793243 [ 5514.003106] LustreError: MGC192.168.203.108@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 5514.004849] LustreError: 325430:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) Skipped 3 previous similar messages [ 5514.153887] Lustre: server umount lustre-MDT0001 complete [ 5525.434108] LustreError: 328913:0:(obd_class.h:479:obd_check_dev()) Device 23 not setup [ 5525.438150] LustreError: 328913:0:(obd_class.h:479:obd_check_dev()) Skipped 15 previous similar messages [ 5525.465095] Lustre: server umount lustre-OST0000 complete [ 5536.585716] Lustre: server umount lustre-OST0001 complete [ 5538.462375] Lustre: DEBUG MARKER: SKIP: conf-sanity test_114 skipping SLOW test 114 [ 5539.184424] Lustre: DEBUG MARKER: SKIP: conf-sanity test_115 skipping excluded test 115 [ 5540.088996] Lustre: DEBUG MARKER: == conf-sanity test 116: big size MDT support ============ 16:48:39 (1767390519) [ 5550.977302] Lustre: DEBUG MARKER: oleg308-server.virtnet: executing load_modules_local [ 5555.040958] loop0: detected capacity change from 0 to 1099511627776 [ 5555.223983] SGI XFS with ACLs, security attributes, quota, no debug enabled [ 5555.226554] *** VALIDATE xfs *** [ 5555.236740] XFS (loop0): Mounting V5 Filesystem [ 5555.251823] XFS (loop0): Ending clean mount [ 5555.889891] loop1: detected capacity change from 0 to 18691697672192 [ 5571.176198] LDISKFS-fs (loop1): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 5579.436622] Lustre: DEBUG MARKER: oleg308-server.virtnet: executing set_hostid [ 5583.928747] Lustre: DEBUG MARKER: oleg308-server.virtnet: executing load_modules_local [ 5588.079659] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 5591.335622] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 5594.316083] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 5597.230560] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 5600.709160] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 5600.757568] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 5600.903606] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 5600.924420] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 5600.983966] Lustre: lustre-MDT0000: new disk, initializing [ 5601.026172] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 5601.030768] Lustre: Skipped 1 previous similar message [ 5601.041929] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 5602.955459] Lustre: DEBUG MARKER: oleg308-server.virtnet: executing set_default_debug -1 all [ 5608.651722] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 5608.698200] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 5608.757066] Lustre: 333483:0:(mgs_llog.c:1348:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 5608.805536] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [ 5608.809536] Lustre: Skipped 1 previous similar message [ 5608.867224] Lustre: lustre-MDT0001: new disk, initializing [ 5608.950140] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [ 5608.958038] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 5610.774095] Lustre: DEBUG MARKER: oleg308-server.virtnet: executing set_default_debug -1 all [ 5614.786309] Lustre: DEBUG MARKER: oleg308-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 5616.608269] Lustre: DEBUG MARKER: oleg308-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 5619.977488] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 5620.017652] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 5620.168965] Lustre: lustre-OST0000: new disk, initializing [ 5620.171400] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 5622.097771] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [ 5622.104386] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 5622.168537] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 5623.020714] Lustre: DEBUG MARKER: oleg308-server.virtnet: executing set_default_debug -1 all [ 5627.176598] Lustre: DEBUG MARKER: oleg308-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 5629.155769] Lustre: DEBUG MARKER: oleg308-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 [ 5629.285799] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [ 5631.150293] Lustre: DEBUG MARKER: oleg308-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid 50 [ 5631.275789] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [ 5632.486893] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5632.488239] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 5632.494825] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 5632.497382] LustreError: Skipped 1 previous similar message [ 5632.501584] Lustre: Skipped 5 previous similar messages [ 5638.322164] LustreError: 336426:0:(obd_class.h:479:obd_check_dev()) Device 19 not setup [ 5638.326353] LustreError: 336426:0:(obd_class.h:479:obd_check_dev()) Skipped 9 previous similar messages [ 5638.390534] Lustre: server umount lustre-OST0000 complete [ 5642.723756] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5642.725906] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 5642.731683] Lustre: Skipped 2 previous similar messages [ 5642.735334] Lustre: Skipped 2 previous similar messages [ 5646.456520] Lustre: server umount lustre-MDT0000 complete [ 5647.841467] LustreError: 333492:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 5647.851498] LustreError: 333492:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 1 previous similar message [ 5648.285331] LustreError: 334266:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1767390628 with bad export cookie 9335818997705794797 [ 5648.295565] LustreError: MGC192.168.203.108@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 5648.480533] Lustre: server umount lustre-MDT0001 complete [ 5652.448536] XFS (loop0): Unmounting Filesystem [ 5654.551714] Lustre: DEBUG MARKER: == conf-sanity test 117: lctl get_param return errors properly ========================================================== 16:50:33 (1767390633) [ 5657.928435] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 5658.185861] LustreError: 337645:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 5658.228794] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 5658.232810] Lustre: Skipped 2 previous similar messages [ 5660.125505] Lustre: DEBUG MARKER: oleg308-server.virtnet: executing set_default_debug -1 all [ 5664.329455] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 5666.427759] Lustre: DEBUG MARKER: oleg308-server.virtnet: executing set_default_debug -1 all [ 5668.993947] Lustre: DEBUG MARKER: oleg308-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 5670.648691] Lustre: DEBUG MARKER: oleg308-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 5673.653444] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 5676.503444] Lustre: DEBUG MARKER: oleg308-server.virtnet: executing set_default_debug -1 all [ 5679.142942] Lustre: DEBUG MARKER: oleg308-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 5688.292771] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 5688.297185] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5688.300705] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 5688.303812] Lustre: Skipped 1 previous similar message [ 5694.260180] LustreError: 339987:0:(obd_class.h:479:obd_check_dev()) Device 21 not setup [ 5694.264373] LustreError: 339987:0:(obd_class.h:479:obd_check_dev()) Skipped 17 previous similar messages [ 5694.326533] Lustre: server umount lustre-OST0000 complete [ 5702.337861] Lustre: server umount lustre-MDT0000 complete [ 5703.648937] LustreError: 337645:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 5703.657413] LustreError: 337645:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 2 previous similar messages [ 5704.127356] LustreError: 337626:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1767390684 with bad export cookie 9335818997705795679 [ 5704.131046] LustreError: 337626:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) Skipped 8 previous similar messages [ 5704.131240] LustreError: MGC192.168.203.108@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 5704.244213] Lustre: server umount lustre-MDT0001 complete [ 5708.819743] Lustre: DEBUG MARKER: oleg308-server.virtnet: executing unload_modules_local [ 5710.184736] Key type lgssc unregistered [ 5710.358637] LNet: 340923:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 5710.363319] LNetError: 340923:0:(acceptor.c:264:lnet_acceptor_remove_socket()) Interface ens2 not found [ 5710.373576] LNet: Removed LNI 192.168.203.108@tcp [ 5710.844168] Key type .llcrypt unregistered [ 5710.846271] Key type ._llcrypt unregistered [ 5724.465866] Key type ._llcrypt registered [ 5724.467156] Key type .llcrypt registered [ 5724.522169] Lustre: DEBUG MARKER: oleg308-server.virtnet: executing load_modules_local [ 5725.059180] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 5725.075528] alg: No test for adler32 (adler32-zlib) [ 5725.978541] Lustre: Lustre: Build Version: 2.17.0_RC4_2_g45eb11f [ 5726.128336] LNet: Added LNI 192.168.203.108@tcp [8/256/0/180] [ 5727.743166] Key type lgssc registered [ 5728.406367] Lustre: Echo OBD driver; http://www.lustre.org/ [ 5732.428606] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 5732.442086] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 5733.686436] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 5735.307473] Lustre: DEBUG MARKER: oleg308-server.virtnet: executing set_default_debug -1 all [ 5739.669122] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 5739.847549] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 5741.524896] Lustre: DEBUG MARKER: oleg308-server.virtnet: executing set_default_debug -1 all [ 5744.061506] Lustre: DEBUG MARKER: oleg308-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 5745.788922] Lustre: DEBUG MARKER: oleg308-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 5749.075689] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 5749.280300] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 5751.998873] Lustre: DEBUG MARKER: oleg308-server.virtnet: executing set_default_debug -1 all [ 5754.697217] Lustre: DEBUG MARKER: oleg308-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 5763.556138] Lustre: lustre-MDT0001-osp-MDT0000: Connection to lustre-MDT0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5763.561108] Lustre: Skipped 1 previous similar message [ 5763.561348] Lustre: lustre-MDT0001: Not available for connect from 0@lo (stopping) [ 5763.566928] Lustre: Skipped 1 previous similar message [ 5765.572187] Lustre: lustre-MDT0001: Not available for connect from 192.168.203.8@tcp (stopping) [ 5768.674770] Lustre: lustre-MDT0001: Not available for connect from 0@lo (stopping) [ 5769.554962] LustreError: 344741:0:(obd_class.h:479:obd_check_dev()) Device 16 not setup [ 5769.591923] Lustre: server umount lustre-MDT0001 complete [ 5770.679768] LustreError: 342443:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0001: not available for connect from 192.168.203.8@tcp (no target). If you are running an HA pair check that the target is mounted on the other server. [ 5770.690284] LustreError: 342443:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 2 previous similar messages [ 5771.033765] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 5773.796835] LustreError: 343576:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 5773.806958] LustreError: 343576:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 1 previous similar message [ 5774.724551] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 5774.771148] Lustre: MGS: Regenerating lustre-MDT0001 log by user request: rc = 0 [ 5774.774782] Lustre: Found index 1 for lustre-MDT0001, updating log [ 5774.796283] Lustre: 342438:0:(mgs_llog.c:1348:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 5774.953934] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 5776.801365] Lustre: DEBUG MARKER: oleg308-server.virtnet: executing set_default_debug -1 all [ 5780.468230] Lustre: lustre-MDT0001-lwp-OST0000: Connection restored to 0@lo (at 0@lo) [ 5780.970516] LustreError: 345771:0:(obd_mount.c:188:lustre_start_simple()) lustre-MDT0001-lwp-OST0000 attach error -17 [ 5780.975733] LustreError: 345771:0:(tgt_mount.c:669:lustre_lwp_setup()) lustre-MDT0001-lwp-OST0000: setup up failed: rc -17 [ 5780.981829] LustreError: 342441:0:(mgc_request.c:614:do_requeue()) failed processing log: -17 [ 5781.015440] LustreError: 345774:0:(obd_config.c:2001:class_config_llog_handler()) MGC192.168.203.108@tcp: cfg command failed: rc = -17 [ 5781.021648] Lustre: cmd=cf001 0:lustre-MDT0001-osp-MDT0000 1:osp 2:lustre-MDT0000-mdtlov_UUID [ 5781.021648] [ 6082.533575] Lustre: lustre-MDT0001-osp-MDT0000: Connection to lustre-MDT0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 6082.533682] LustreError: lustre-MDT0001-osp-MDT0000: operation mds_statfs to node 0@lo failed: rc = -107 [ 6082.549127] Lustre: lustre-MDT0001: Not available for connect from 0@lo (stopping) [ 6082.552893] Lustre: Skipped 2 previous similar messages [ 6087.651606] Lustre: lustre-MDT0001: Not available for connect from 0@lo (stopping) [ 6087.655655] Lustre: Skipped 2 previous similar messages [ 6094.303165] Lustre: lustre-MDT0001 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 6094.359855] LustreError: 356311:0:(obd_class.h:479:obd_check_dev()) Device 16 not setup [ 6094.363157] LustreError: 356311:0:(obd_class.h:479:obd_check_dev()) Skipped 5 previous similar messages [ 6094.395740] Lustre: server umount lustre-MDT0001 complete [ 6095.778670] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 6097.890613] LustreError: 342443:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 6097.900322] LustreError: 342443:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 1 previous similar message [ 6098.751024] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 6098.806982] Lustre: MGS: Regenerating lustre-MDT0001 log by user request: rc = 0 [ 6098.809551] Lustre: Found index 1 for lustre-MDT0001, updating log [ 6098.831357] Lustre: 342437:0:(mgs_llog.c:1348:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 6099.003573] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 6100.730361] Lustre: DEBUG MARKER: oleg308-server.virtnet: executing set_default_debug -1 all [ 6104.037188] Lustre: lustre-MDT0001-lwp-OST0000: Connection restored to 0@lo (at 0@lo) [ 6107.627246] LustreError: 357440:0:(obd_mount.c:188:lustre_start_simple()) lustre-MDT0001-lwp-OST0000 attach error -17 [ 6107.632299] LustreError: 357440:0:(tgt_mount.c:669:lustre_lwp_setup()) lustre-MDT0001-lwp-OST0000: setup up failed: rc -17 [ 6107.637138] LustreError: 342441:0:(mgc_request.c:614:do_requeue()) failed processing log: -17 [ 6107.641221] LustreError: 342441:0:(mgc_request.c:614:do_requeue()) Skipped 1 previous similar message [ 6107.673490] LustreError: 357448:0:(obd_config.c:689:class_setup()) Device 8 already setup (type osp) [ 6107.677919] LustreError: 357448:0:(obd_config.c:797:class_detach()) OBD device 8 still set up [ 6107.681870] LustreError: 357448:0:(obd_config.c:2001:class_config_llog_handler()) MGC192.168.203.108@tcp: cfg command failed: rc = -17 [ 6107.687352] Lustre: cmd=cf003 0:lustre-MDT0001-osp-MDT0000 1:lustre-MDT0001_UUID 2:192.168.203.108@tcp [ 6107.687352] [ 6405.569335] Lustre: lustre-MDT0001: Not available for connect from 192.168.203.8@tcp (stopping) [ 6405.575448] Lustre: Skipped 4 previous similar messages [ 6406.115259] LustreError: lustre-MDT0001-osp-MDT0000: operation mds_statfs to node 0@lo failed: rc = -107 [ 6406.115462] Lustre: lustre-MDT0001-osp-MDT0000: Connection to lustre-MDT0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 6406.128169] Lustre: Skipped 1 previous similar message [ 6410.442746] LustreError: 367939:0:(obd_class.h:479:obd_check_dev()) Device 16 not setup [ 6410.446207] LustreError: 367939:0:(obd_class.h:479:obd_check_dev()) Skipped 5 previous similar messages [ 6410.475735] Lustre: server umount lustre-MDT0001 complete [ 6410.677652] LustreError: 342445:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0001: not available for connect from 192.168.203.8@tcp (no target). If you are running an HA pair check that the target is mounted on the other server. [ 6410.682865] LustreError: 342445:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 1 previous similar message [ 6411.770876] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 6415.397439] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 6415.443529] Lustre: MGS: Regenerating lustre-MDT0001 log by user request: rc = 0 [ 6415.445760] Lustre: Found index 1 for lustre-MDT0001, updating log [ 6415.461522] Lustre: 342438:0:(mgs_llog.c:1348:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 6415.603171] Lustre: lustre-MDT0001: Imperative Recovery enabled, recovery window shrunk from 60-180 down to 60-180 [ 6417.223484] Lustre: DEBUG MARKER: oleg308-server.virtnet: executing set_default_debug -1 all [ 6420.972611] Lustre: lustre-MDT0001-lwp-OST0000: Connection restored to 0@lo (at 0@lo) [ 6423.019481] LustreError: 369025:0:(obd_mount.c:188:lustre_start_simple()) lustre-MDT0001-lwp-OST0000 attach error -17 [ 6423.027075] LustreError: 369025:0:(tgt_mount.c:669:lustre_lwp_setup()) lustre-MDT0001-lwp-OST0000: setup up failed: rc -17 [ 6423.034884] LustreError: 342441:0:(mgc_request.c:614:do_requeue()) failed processing log: -17 [ 6423.039130] LustreError: 342441:0:(mgc_request.c:614:do_requeue()) Skipped 1 previous similar message [ 6423.073376] LustreError: 369032:0:(lod_lov.c:212:lod_add_device()) lustre-MDT0001-osp-MDT0000: cannot connect to next dev lustre-MDT0001_UUID (-114) [ 6423.083094] LustreError: 369032:0:(obd_config.c:2001:class_config_llog_handler()) MGC192.168.203.108@tcp: cfg command failed: rc = -114 [ 6423.088327] Lustre: cmd=cf014 0:lustre-MDT0000-mdtlov 1:lustre-MDT0001_UUID 2:1 3:1 [ 6423.088327] [ 6733.281159] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 6733.286224] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 6733.293993] Lustre: Skipped 1 previous similar message [ 6733.296647] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 6733.300363] Lustre: Skipped 2 previous similar messages [ 6738.309318] LustreError: 379673:0:(obd_class.h:479:obd_check_dev()) Device 9 not setup [ 6738.311759] LustreError: 379673:0:(obd_class.h:479:obd_check_dev()) Skipped 5 previous similar messages [ 6738.343695] Lustre: server umount lustre-MDT0000 complete [ 6739.662402] LustreError: 368504:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1767391720 with bad export cookie 16889016550426435971 [ 6739.666381] LustreError: MGC192.168.203.108@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 6739.666538] LustreError: 368504:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) Skipped 2 previous similar messages [ 6741.471895] LustreError: 342445:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 6741.472240] Lustre: lustre-MDT0001-lwp-OST0000: Connection to lustre-MDT0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 6741.476580] LustreError: 342445:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 2 previous similar messages [ 6741.480719] Lustre: Skipped 2 previous similar messages [ 6745.795011] LustreError: 379874:0:(obd_class.h:479:obd_check_dev()) Device 16 not setup [ 6745.797758] LustreError: 379874:0:(obd_class.h:479:obd_check_dev()) Skipped 5 previous similar messages [ 6745.853021] Lustre: server umount lustre-MDT0001 complete [ 6747.522962] Lustre: server umount lustre-OST0000 complete [ 6751.610731] Lustre: DEBUG MARKER: oleg308-server.virtnet: executing set_hostid [ 6755.975888] Lustre: DEBUG MARKER: oleg308-server.virtnet: executing load_modules_local [ 6759.837661] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 6762.974797] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 6766.047319] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 6769.226462] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 6772.654481] Lustre: DEBUG MARKER: == conf-sanity test 120: cross-target rename should not create bad symlinks ========================================================== 17:09:11 (1767391751) [ 6775.900549] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 6775.931739] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 6776.062760] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 6776.081385] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 6776.144177] Lustre: lustre-MDT0000: new disk, initializing [ 6776.181867] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 6776.192150] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 6778.075776] Lustre: DEBUG MARKER: oleg308-server.virtnet: executing set_default_debug -1 all [ 6783.582127] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 6783.622987] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 6783.674953] Lustre: 382607:0:(mgs_llog.c:1348:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 6783.708722] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [ 6783.712306] Lustre: Skipped 1 previous similar message [ 6783.764165] Lustre: lustre-MDT0001: new disk, initializing [ 6783.798224] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 6783.810754] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [ 6783.814139] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 6785.107586] Lustre: DEBUG MARKER: oleg308-server.virtnet: executing set_default_debug -1 all [ 6789.028914] Lustre: DEBUG MARKER: oleg308-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 6790.720929] Lustre: DEBUG MARKER: oleg308-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 6793.816302] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 6793.856882] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 6794.014549] Lustre: lustre-OST0000: new disk, initializing [ 6794.018345] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 6794.070127] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 6796.813631] Lustre: DEBUG MARKER: oleg308-server.virtnet: executing set_default_debug -1 all [ 6801.002217] Lustre: DEBUG MARKER: oleg308-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 6801.933334] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [ 6801.940375] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 6801.995746] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 6809.567934] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 6809.570552] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 6809.575212] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 6809.578909] Lustre: Skipped 5 previous similar messages [ 6815.698095] LustreError: 385053:0:(obd_class.h:479:obd_check_dev()) Device 22 not setup [ 6815.701293] LustreError: 385053:0:(obd_class.h:479:obd_check_dev()) Skipped 11 previous similar messages [ 6815.758205] Lustre: server umount lustre-MDT0000 complete [ 6817.253306] LustreError: 382618:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 6817.263619] LustreError: 382618:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 3 previous similar messages [ 6817.555156] LustreError: 382600:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1767391798 with bad export cookie 16889016550426437924 [ 6817.559745] LustreError: 383399:0:(osp_object.c:617:osp_attr_get()) lustre-MDT0000-osp-MDT0001: osp_attr_get update error [0x200000401:0x2:0x0]: rc = -5 [ 6817.560796] LustreError: MGC192.168.203.108@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 6817.563823] LustreError: 383399:0:(client.c:1380:ptlrpc_import_delay_req()) @@@ IMP_CLOSED req@00000000229ba7fd x1853243477308160/t0(0) o1000->lustre-MDT0000-osp-MDT0001@0@lo:24/4 lens 264/4320 e 0 to 0 dl 0 ref 2 fl Rpc:QU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 [ 6817.693702] Lustre: server umount lustre-MDT0001 complete [ 6829.623463] Lustre: server umount lustre-OST0000 complete [ 6842.667606] Lustre: DEBUG MARKER: oleg308-server.virtnet: executing set_hostid [ 6846.733622] Lustre: DEBUG MARKER: oleg308-server.virtnet: executing load_modules_local [ 6851.228070] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 6854.189082] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 6857.259355] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 6860.575899] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 6861.228527] Lustre: DEBUG MARKER: == conf-sanity test complete, duration 6742 sec ========== 17:10:40 (1767391840) [ 6861.779236] Lustre: DEBUG MARKER: === conf-sanity: start cleanup 17:10:41 (1767391841) === [ 6863.079853] Lustre: DEBUG MARKER: === conf-sanity: finish cleanup 17:10:42 (1767391842) === [ 6880.261294] Lustre: DEBUG MARKER: oleg308-server.virtnet: executing unload_modules_local [ 6881.775510] Key type lgssc unregistered [ 6881.942616] LNet: 390559:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 6881.947987] LNetError: 390559:0:(acceptor.c:264:lnet_acceptor_remove_socket()) Interface ens2 not found [ 6881.960555] LNet: Removed LNI 192.168.203.108@tcp [ 6882.436306] Key type .llcrypt unregistered [ 6882.438247] Key type ._llcrypt unregistered