[ 0.000000] Linux version 4.18.0rh8.10-debug (green@maintenance) (gcc version 8.5.0 20210514 (Red Hat 8.5.0-26) (GCC)) #2 SMP Mon Jul 14 01:24:22 EDT 2025 [ 0.000000] Command line: rd.shell root=nbd:192.168.200.253:rocky8.10:ext4:ro:-p,-b4096 ro crashkernel=256M panic=1 nomodeset ipmtu=9000 ip=dhcp rd.neednet=1 init_on_free=off mitigations=off console=ttyS1,115200 audit=0 [ 0.000000] x86/fpu: Supporting XSAVE feature 0x001: 'x87 floating point registers' [ 0.000000] x86/fpu: Supporting XSAVE feature 0x002: 'SSE registers' [ 0.000000] x86/fpu: Supporting XSAVE feature 0x004: 'AVX registers' [ 0.000000] x86/fpu: xstate_offset[2]: 576, xstate_sizes[2]: 256 [ 0.000000] x86/fpu: Enabled xstate features 0x7, context size is 832 bytes, using 'standard' format. [ 0.000000] signal: max sigframe size: 1776 [ 0.000000] BIOS-provided physical RAM map: [ 0.000000] BIOS-e820: [mem 0x0000000000000000-0x000000000009fbff] usable [ 0.000000] BIOS-e820: [mem 0x000000000009fc00-0x000000000009ffff] reserved [ 0.000000] BIOS-e820: [mem 0x00000000000f0000-0x00000000000fffff] reserved [ 0.000000] BIOS-e820: [mem 0x0000000000100000-0x00000000bffcdfff] usable [ 0.000000] BIOS-e820: [mem 0x00000000bffce000-0x00000000bfffffff] reserved [ 0.000000] BIOS-e820: [mem 0x00000000feffc000-0x00000000feffffff] reserved [ 0.000000] BIOS-e820: [mem 0x00000000fffc0000-0x00000000ffffffff] reserved [ 0.000000] BIOS-e820: [mem 0x0000000100000000-0x0000000146dfffff] usable [ 0.000000] NX (Execute Disable) protection: active [ 0.000000] SMBIOS 3.0.0 present. [ 0.000000] DMI: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.16.3-2.fc40 04/01/2014 [ 0.000000] Hypervisor detected: KVM [ 0.000000] kvm-clock: Using msrs 4b564d01 and 4b564d00 [ 0.000000] kvm-clock: using sched offset of 488195326 cycles [ 0.000000] clocksource: kvm-clock: mask: 0xffffffffffffffff max_cycles: 0x1cd42e4dffb, max_idle_ns: 881590591483 ns [ 0.000000] tsc: Detected 2399.998 MHz processor [ 0.000000] last_pfn = 0x146e00 max_arch_pfn = 0x400000000 [ 0.000000] x86/PAT: Configuration [0-7]: WB WC UC- UC WB WP UC- WT [ 0.000000] last_pfn = 0xbffce max_arch_pfn = 0x400000000 [ 0.000000] found SMP MP-table at [mem 0x000f53f0-0x000f53ff] [ 0.000000] RAMDISK: [mem 0xbcc54000-0xbffbffff] [ 0.000000] ACPI: Early table checksum verification disabled [ 0.000000] ACPI: RSDP 0x00000000000F5200 000014 (v00 BOCHS ) [ 0.000000] ACPI: RSDT 0x00000000BFFE1D87 000034 (v01 BOCHS BXPC 00000001 BXPC 00000001) [ 0.000000] ACPI: FACP 0x00000000BFFE1C23 000074 (v01 BOCHS BXPC 00000001 BXPC 00000001) [ 0.000000] ACPI: DSDT 0x00000000BFFE0040 001BE3 (v01 BOCHS BXPC 00000001 BXPC 00000001) [ 0.000000] ACPI: FACS 0x00000000BFFE0000 000040 [ 0.000000] ACPI: APIC 0x00000000BFFE1C97 000090 (v03 BOCHS BXPC 00000001 BXPC 00000001) [ 0.000000] ACPI: HPET 0x00000000BFFE1D27 000038 (v01 BOCHS BXPC 00000001 BXPC 00000001) [ 0.000000] ACPI: WAET 0x00000000BFFE1D5F 000028 (v01 BOCHS BXPC 00000001 BXPC 00000001) [ 0.000000] ACPI: Reserving FACP table memory at [mem 0xbffe1c23-0xbffe1c96] [ 0.000000] ACPI: Reserving DSDT table memory at [mem 0xbffe0040-0xbffe1c22] [ 0.000000] ACPI: Reserving FACS table memory at [mem 0xbffe0000-0xbffe003f] [ 0.000000] ACPI: Reserving APIC table memory at [mem 0xbffe1c97-0xbffe1d26] [ 0.000000] ACPI: Reserving HPET table memory at [mem 0xbffe1d27-0xbffe1d5e] [ 0.000000] ACPI: Reserving WAET table memory at [mem 0xbffe1d5f-0xbffe1d86] [ 0.000000] No NUMA configuration found [ 0.000000] Faking a node at [mem 0x0000000000000000-0x0000000146dfffff] [ 0.000000] NODE_DATA(0) allocated [mem 0x1465a3000-0x1465cdfff] [ 0.000000] Reserving 256MB of memory at 2752MB for crashkernel (System RAM: 4205MB) [ 0.000000] Zone ranges: [ 0.000000] DMA [mem 0x0000000000001000-0x0000000000ffffff] [ 0.000000] DMA32 [mem 0x0000000001000000-0x00000000ffffffff] [ 0.000000] Normal [mem 0x0000000100000000-0x0000000146dfffff] [ 0.000000] Device empty [ 0.000000] Movable zone start for each node [ 0.000000] Early memory node ranges [ 0.000000] node 0: [mem 0x0000000000001000-0x000000000009efff] [ 0.000000] node 0: [mem 0x0000000000100000-0x00000000bffcdfff] [ 0.000000] node 0: [mem 0x0000000100000000-0x0000000146dfffff] [ 0.000000] Zeroed struct page in unavailable ranges: 4756 pages [ 0.000000] Initmem setup node 0 [mem 0x0000000000001000-0x0000000146dfffff] [ 0.000000] ACPI: PM-Timer IO Port: 0x608 [ 0.000000] ACPI: LAPIC_NMI (acpi_id[0xff] dfl dfl lint[0x1]) [ 0.000000] IOAPIC[0]: apic_id 0, version 17, address 0xfec00000, GSI 0-23 [ 0.000000] ACPI: INT_SRC_OVR (bus 0 bus_irq 0 global_irq 2 dfl dfl) [ 0.000000] ACPI: INT_SRC_OVR (bus 0 bus_irq 5 global_irq 5 high level) [ 0.000000] ACPI: INT_SRC_OVR (bus 0 bus_irq 9 global_irq 9 high level) [ 0.000000] ACPI: INT_SRC_OVR (bus 0 bus_irq 10 global_irq 10 high level) [ 0.000000] ACPI: INT_SRC_OVR (bus 0 bus_irq 11 global_irq 11 high level) [ 0.000000] Using ACPI (MADT) for SMP configuration information [ 0.000000] ACPI: HPET id: 0x8086a201 base: 0xfed00000 [ 0.000000] TSC deadline timer available [ 0.000000] smpboot: Allowing 4 CPUs, 0 hotplug CPUs [ 0.000000] kvm-guest: KVM setup pv remote TLB flush [ 0.000000] kvm-guest: setup PV sched yield [ 0.000000] PM: Registered nosave memory: [mem 0x00000000-0x00000fff] [ 0.000000] PM: Registered nosave memory: [mem 0x0009f000-0x0009ffff] [ 0.000000] PM: Registered nosave memory: [mem 0x000a0000-0x000effff] [ 0.000000] PM: Registered nosave memory: [mem 0x000f0000-0x000fffff] [ 0.000000] PM: Registered nosave memory: [mem 0xbffce000-0xbfffffff] [ 0.000000] PM: Registered nosave memory: [mem 0xc0000000-0xfeffbfff] [ 0.000000] PM: Registered nosave memory: [mem 0xfeffc000-0xfeffffff] [ 0.000000] PM: Registered nosave memory: [mem 0xff000000-0xfffbffff] [ 0.000000] PM: Registered nosave memory: [mem 0xfffc0000-0xffffffff] [ 0.000000] [mem 0xc0000000-0xfeffbfff] available for PCI devices [ 0.000000] Booting paravirtualized kernel on KVM [ 0.000000] clocksource: refined-jiffies: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 1910969940391419 ns [ 0.000000] setup_percpu: NR_CPUS:8192 nr_cpumask_bits:4 nr_cpu_ids:4 nr_node_ids:1 [ 0.000000] percpu: Embedded 63 pages/cpu s221184 r8192 d28672 u524288 [ 0.000000] kvm-guest: PV spinlocks enabled [ 0.000000] PV qspinlock hash table entries: 256 (order: 0, 4096 bytes, linear) [ 0.000000] Built 1 zonelists, mobility grouping on. Total pages: 1059606 [ 0.000000] Policy zone: Normal [ 0.000000] Kernel command line: rd.shell root=nbd:192.168.200.253:rocky8.10:ext4:ro:-p,-b4096 ro crashkernel=256M panic=1 nomodeset ipmtu=9000 ip=dhcp rd.neednet=1 init_on_free=off mitigations=off console=ttyS1,115200 audit=0 [ 0.000000] Specific versions of hardware are certified with Red Hat Enterprise Linux 8. Please see the list of hardware certified with Red Hat Enterprise Linux 8 at https://catalog.redhat.com. [ 0.000000] audit: disabled (until reboot) [ 0.000000] software IO TLB: area num 4. [ 0.000000] Memory: 2829652K/4306352K available (18435K kernel code, 11221K rwdata, 7248K rodata, 2908K init, 18040K bss, 524584K reserved, 0K cma-reserved) [ 0.000000] SLUB: HWalign=64, Order=0-3, MinObjects=0, CPUs=4, Nodes=1 [ 0.000000] kmemleak: Kernel memory leak detector disabled [ 0.000000] ftrace: allocating 41240 entries in 162 pages [ 0.000000] ftrace: allocated 162 pages with 3 groups [ 0.000000] rcu: Hierarchical RCU implementation. [ 0.000000] rcu: RCU event tracing is enabled. [ 0.000000] rcu: RCU restricting CPUs from NR_CPUS=8192 to nr_cpu_ids=4. [ 0.000000] rcu: RCU callback double-/use-after-free debug enabled. [ 0.000000] Rude variant of Tasks RCU enabled. [ 0.000000] Tracing variant of Tasks RCU enabled. [ 0.000000] rcu: RCU calculated value of scheduler-enlistment delay is 100 jiffies. [ 0.000000] rcu: Adjusting geometry for rcu_fanout_leaf=16, nr_cpu_ids=4 [ 0.000000] NR_IRQS: 524544, nr_irqs: 456, preallocated irqs: 16 [ 0.000000] random: get_random_bytes called from start_kernel+0x622/0x9a8 with crng_init=0 [ 0.001000] Console: colour *CGA 80x25 [ 0.001000] printk: console [ttyS1] enabled [ 0.001000] ACPI: Core revision 20220331 [ 0.001000] clocksource: hpet: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 19112604467 ns [ 0.001012] APIC: Switch to symmetric I/O mode setup [ 0.002343] x2apic enabled [ 0.003000] Switched APIC routing to physical x2apic. [ 0.003017] kvm-guest: setup PV IPIs [ 0.005896] ..TIMER: vector=0x30 apic1=0 pin1=2 apic2=-1 pin2=-1 [ 0.006000] clocksource: tsc-early: mask: 0xffffffffffffffff max_cycles: 0x229835b7123, max_idle_ns: 440795242976 ns [ 0.006031] Calibrating delay loop (skipped) preset value.. 4799.99 BogoMIPS (lpj=2399998) [ 0.007013] pid_max: default: 32768 minimum: 301 [ 0.008156] LSM: Security Framework initializing [ 0.009000] Yama: becoming mindful. [ 0.009037] SELinux: Initializing. [ 0.010000] *** VALIDATE selinux *** [ 0.012000] Dentry cache hash table entries: 1048576 (order: 11, 8388608 bytes, vmalloc) [ 0.016683] Inode-cache hash table entries: 524288 (order: 10, 4194304 bytes, vmalloc) [ 0.018134] Mount-cache hash table entries: 16384 (order: 5, 131072 bytes, vmalloc) [ 0.019132] Mountpoint-cache hash table entries: 16384 (order: 5, 131072 bytes, vmalloc) [ 0.021031] *** VALIDATE tmpfs *** [ 0.022424] *** VALIDATE proc *** [ 0.024243] *** VALIDATE cgroup *** [ 0.025000] *** VALIDATE cgroup2 *** [ 0.025280] x86/cpu: User Mode Instruction Prevention (UMIP) activated [ 0.026158] Last level iTLB entries: 4KB 0, 2MB 0, 4MB 0 [ 0.027013] Last level dTLB entries: 4KB 0, 2MB 0, 4MB 0, 1GB 0 [ 0.029030] Spectre V2 : User space: Vulnerable [ 0.030013] Speculative Store Bypass: Vulnerable [ 0.033098] debug: unmapping init [mem 0xffffffffb4259000-0xffffffffb4260fff] [ 0.036202] smpboot: CPU0: Intel(R) Xeon(R) CPU E5-2695 v2 @ 2.40GHz (family: 0x6, model: 0x3e, stepping: 0x4) [ 0.037705] Performance Events: IvyBridge events, full-width counters, Intel PMU driver. [ 0.038025] ... version: 2 [ 0.039012] ... bit width: 48 [ 0.040013] ... generic registers: 4 [ 0.041017] ... value mask: 0000ffffffffffff [ 0.042012] ... max period: 00007fffffffffff [ 0.043016] ... fixed-purpose events: 3 [ 0.044015] ... event mask: 000000070000000f [ 0.045302] rcu: Hierarchical SRCU implementation. [ 0.047548] smp: Bringing up secondary CPUs ... [ 0.048639] x86: Booting SMP configuration: [ 0.049030] .... node #0, CPUs: #1 #2 #3 [ 0.061267] smp: Brought up 1 node, 4 CPUs [ 0.063014] smpboot: Max logical packages: 1 [ 0.064011] smpboot: Total of 4 processors activated (19199.98 BogoMIPS) [ 0.246028] node 0 deferred pages initialised in 177ms [ 0.248108] devtmpfs: initialized [ 0.250274] x86/mm: Memory block size: 128MB [ 0.252000] gcov: version magic: 0x41383552 [ 0.255383] clocksource: jiffies: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 1911260446275000 ns [ 0.257083] futex hash table entries: 1024 (order: 4, 65536 bytes, vmalloc) [ 0.259436] pinctrl core: initialized pinctrl subsystem [ 0.261264] [ 0.261830] ************************************************************* [ 0.265013] ** NOTICE NOTICE NOTICE NOTICE NOTICE NOTICE NOTICE ** [ 0.267011] ** ** [ 0.269017] ** IOMMU DebugFS SUPPORT HAS BEEN ENABLED IN THIS KERNEL ** [ 0.271014] ** ** [ 0.273014] ** This means that this kernel is built to expose internal ** [ 0.276015] ** IOMMU data structures, which may compromise security on ** [ 0.278011] ** your system. ** [ 0.281013] ** ** [ 0.283010] ** If you see this message and you are not debugging the ** [ 0.285013] ** kernel, report this immediately to your vendor! ** [ 0.287014] ** ** [ 0.289014] ** NOTICE NOTICE NOTICE NOTICE NOTICE NOTICE NOTICE ** [ 0.292013] ************************************************************* [ 0.294733] NET: Registered protocol family 16 [ 0.296504] DMA: preallocated 512 KiB GFP_KERNEL pool for atomic allocations [ 0.299061] DMA: preallocated 512 KiB GFP_KERNEL|GFP_DMA pool for atomic allocations [ 0.302064] DMA: preallocated 512 KiB GFP_KERNEL|GFP_DMA32 pool for atomic allocations [ 0.306100] cpuidle: using governor menu [ 0.307787] acpiphp: ACPI Hot Plug PCI Controller Driver version: 0.5 [ 0.310572] PCI: Using configuration type 1 for base access [ 0.313370] core: PMU erratum BJ122, BV98, HSD29 worked around, HT is on [ 0.323157] HugeTLB registered 1.00 GiB page size, pre-allocated 0 pages [ 0.327119] HugeTLB registered 2.00 MiB page size, pre-allocated 0 pages [ 0.330165] cryptd: max_cpu_qlen set to 1000 [ 0.332210] ACPI: Added _OSI(Module Device) [ 0.333000] ACPI: Added _OSI(Processor Device) [ 0.333000] ACPI: Added _OSI(3.0 _SCP Extensions) [ 0.335024] ACPI: Added _OSI(Processor Aggregator Device) [ 0.339863] ACPI: 1 ACPI AML tables successfully acquired and loaded [ 0.347012] ACPI: Interpreter enabled [ 0.348084] ACPI: PM: (supports S0 S3 S4 S5) [ 0.350024] ACPI: Using IOAPIC for interrupt routing [ 0.352422] PCI: Using host bridge windows from ACPI; if necessary, use "pci=nocrs" and report a bug [ 0.356482] ACPI: Enabled 2 GPEs in block 00 to 0F [ 0.365763] ACPI: PCI Root Bridge [PCI0] (domain 0000 [bus 00-ff]) [ 0.368062] acpi PNP0A03:00: _OSC: OS supports [ASPM ClockPM Segments MSI HPX-Type3] [ 0.371032] acpi PNP0A03:00: _OSC: not requesting OS control; OS requires [ExtendedConfig ASPM ClockPM MSI] [ 0.375114] acpi PNP0A03:00: fail to add MMCONFIG information, can't access extended PCI configuration space under this bridge. [ 0.381481] acpiphp: Slot [2] registered [ 0.383221] acpiphp: Slot [5] registered [ 0.384154] acpiphp: Slot [6] registered [ 0.386248] acpiphp: Slot [7] registered [ 0.389157] acpiphp: Slot [8] registered [ 0.390133] acpiphp: Slot [9] registered [ 0.392131] acpiphp: Slot [10] registered [ 0.394200] acpiphp: Slot [3] registered [ 0.396135] acpiphp: Slot [4] registered [ 0.397089] acpiphp: Slot [11] registered [ 0.398112] acpiphp: Slot [12] registered [ 0.399081] acpiphp: Slot [13] registered [ 0.400087] acpiphp: Slot [14] registered [ 0.401115] acpiphp: Slot [15] registered [ 0.403103] acpiphp: Slot [16] registered [ 0.404087] acpiphp: Slot [17] registered [ 0.405136] acpiphp: Slot [18] registered [ 0.406090] acpiphp: Slot [19] registered [ 0.408086] acpiphp: Slot [20] registered [ 0.409089] acpiphp: Slot [21] registered [ 0.410100] acpiphp: Slot [22] registered [ 0.411086] acpiphp: Slot [23] registered [ 0.413100] acpiphp: Slot [24] registered [ 0.414118] acpiphp: Slot [25] registered [ 0.415102] acpiphp: Slot [26] registered [ 0.416100] acpiphp: Slot [27] registered [ 0.418107] acpiphp: Slot [28] registered [ 0.419099] acpiphp: Slot [29] registered [ 0.420093] acpiphp: Slot [30] registered [ 0.421110] acpiphp: Slot [31] registered [ 0.423063] PCI host bridge to bus 0000:00 [ 0.424015] pci_bus 0000:00: root bus resource [io 0x0000-0x0cf7 window] [ 0.426023] pci_bus 0000:00: root bus resource [io 0x0d00-0xffff window] [ 0.428039] pci_bus 0000:00: root bus resource [mem 0x000a0000-0x000bffff window] [ 0.429000] pci_bus 0000:00: root bus resource [mem 0xc0000000-0xfebfffff window] [ 0.431043] pci_bus 0000:00: root bus resource [mem 0x380000000000-0x38007fffffff window] [ 0.434046] pci_bus 0000:00: root bus resource [bus 00-ff] [ 0.437388] pci 0000:00:00.0: [8086:1237] type 00 class 0x060000 [ 0.440597] pci 0000:00:01.0: [8086:7000] type 00 class 0x060100 [ 0.444253] pci 0000:00:01.1: [8086:7010] type 00 class 0x010180 [ 0.454790] pci 0000:00:01.1: reg 0x20: [io 0xc320-0xc32f] [ 0.460608] pci 0000:00:01.1: legacy IDE quirk: reg 0x10: [io 0x01f0-0x01f7] [ 0.463028] pci 0000:00:01.1: legacy IDE quirk: reg 0x14: [io 0x03f6] [ 0.466021] pci 0000:00:01.1: legacy IDE quirk: reg 0x18: [io 0x0170-0x0177] [ 0.468018] pci 0000:00:01.1: legacy IDE quirk: reg 0x1c: [io 0x0376] [ 0.470641] pci 0000:00:01.3: [8086:7113] type 00 class 0x068000 [ 0.473839] pci 0000:00:01.3: quirk: [io 0x0600-0x063f] claimed by PIIX4 ACPI [ 0.476050] pci 0000:00:01.3: quirk: [io 0x0700-0x070f] claimed by PIIX4 SMB [ 0.478755] pci 0000:00:02.0: [1af4:1000] type 00 class 0x020000 [ 0.484017] pci 0000:00:02.0: reg 0x10: [io 0xc300-0xc31f] [ 0.497017] pci 0000:00:02.0: reg 0x20: [mem 0x380000000000-0x380000003fff 64bit pref] [ 0.504016] pci 0000:00:02.0: reg 0x30: [mem 0xfeb80000-0xfebbffff pref] [ 0.510167] pci 0000:00:05.0: [1af4:1001] type 00 class 0x010000 [ 0.517020] pci 0000:00:05.0: reg 0x10: [io 0xc000-0xc07f] [ 0.523023] pci 0000:00:05.0: reg 0x14: [mem 0xfebc0000-0xfebc0fff] [ 0.542037] pci 0000:00:05.0: reg 0x20: [mem 0x380000004000-0x380000007fff 64bit pref] [ 0.556995] pci 0000:00:06.0: [1af4:1001] type 00 class 0x010000 [ 0.565997] pci 0000:00:06.0: reg 0x10: [io 0xc080-0xc0ff] [ 0.572018] pci 0000:00:06.0: reg 0x14: [mem 0xfebc1000-0xfebc1fff] [ 0.585018] pci 0000:00:06.0: reg 0x20: [mem 0x380000008000-0x38000000bfff 64bit pref] [ 0.596494] pci 0000:00:07.0: [1af4:1001] type 00 class 0x010000 [ 0.603019] pci 0000:00:07.0: reg 0x10: [io 0xc100-0xc17f] [ 0.609017] pci 0000:00:07.0: reg 0x14: [mem 0xfebc2000-0xfebc2fff] [ 0.624018] pci 0000:00:07.0: reg 0x20: [mem 0x38000000c000-0x38000000ffff 64bit pref] [ 0.633882] pci 0000:00:08.0: [1af4:1001] type 00 class 0x010000 [ 0.641017] pci 0000:00:08.0: reg 0x10: [io 0xc180-0xc1ff] [ 0.648016] pci 0000:00:08.0: reg 0x14: [mem 0xfebc3000-0xfebc3fff] [ 0.661035] pci 0000:00:08.0: reg 0x20: [mem 0x380000010000-0x380000013fff 64bit pref] [ 0.670323] pci 0000:00:09.0: [1af4:1001] type 00 class 0x010000 [ 0.672000] pci 0000:00:09.0: reg 0x10: [io 0xc200-0xc27f] [ 0.677018] pci 0000:00:09.0: reg 0x14: [mem 0xfebc4000-0xfebc4fff] [ 0.689022] pci 0000:00:09.0: reg 0x20: [mem 0x380000014000-0x380000017fff 64bit pref] [ 0.700290] pci 0000:00:0a.0: [1af4:1001] type 00 class 0x010000 [ 0.706020] pci 0000:00:0a.0: reg 0x10: [io 0xc280-0xc2ff] [ 0.711020] pci 0000:00:0a.0: reg 0x14: [mem 0xfebc5000-0xfebc5fff] [ 0.725028] pci 0000:00:0a.0: reg 0x20: [mem 0x380000018000-0x38000001bfff 64bit pref] [ 0.737000] ACPI: PCI: Interrupt link LNKA configured for IRQ 10 [ 0.741555] ACPI: PCI: Interrupt link LNKB configured for IRQ 10 [ 0.743687] ACPI: PCI: Interrupt link LNKC configured for IRQ 11 [ 0.746414] ACPI: PCI: Interrupt link LNKD configured for IRQ 11 [ 0.749377] ACPI: PCI: Interrupt link LNKS configured for IRQ 9 [ 0.755063] iommu: Default domain type: Passthrough [ 0.756521] SCSI subsystem initialized [ 0.758199] ACPI: bus type USB registered [ 0.760160] usbcore: registered new interface driver usbfs [ 0.762117] usbcore: registered new interface driver hub [ 0.764121] usbcore: registered new device driver usb [ 0.765213] pps_core: LinuxPPS API ver. 1 registered [ 0.767020] pps_core: Software ver. 5.3.6 - Copyright 2005-2007 Rodolfo Giometti [ 0.771065] PTP clock support registered [ 0.774039] EDAC MC: Ver: 3.0.0 [ 0.776201] PCI: Using ACPI for IRQ routing [ 0.779140] NetLabel: Initializing [ 0.781017] NetLabel: domain hash size = 128 [ 0.783018] NetLabel: protocols = UNLABELED CIPSOv4 CALIPSO [ 0.785103] NetLabel: unlabeled traffic allowed by default [ 0.788143] vgaarb: loaded [ 0.790340] hpet0: at MMIO 0xfed00000, IRQs 2, 8, 0 [ 0.792022] hpet0: 3 comparators, 64-bit 100.000000 MHz counter [ 0.800321] clocksource: Switched to clocksource kvm-clock [ 0.911930] VFS: Disk quotas dquot_6.6.0 [ 0.913804] VFS: Dquot-cache hash table entries: 512 (order 0, 4096 bytes) [ 0.916467] *** VALIDATE ramfs *** [ 0.917773] *** VALIDATE hugetlbfs *** [ 0.919448] pnp: PnP ACPI init [ 0.922797] pnp: PnP ACPI: found 6 devices [ 0.940434] clocksource: acpi_pm: mask: 0xffffff max_cycles: 0xffffff, max_idle_ns: 2085701024 ns [ 0.943945] pci_bus 0000:00: resource 4 [io 0x0000-0x0cf7 window] [ 0.946197] pci_bus 0000:00: resource 5 [io 0x0d00-0xffff window] [ 0.948456] pci_bus 0000:00: resource 6 [mem 0x000a0000-0x000bffff window] [ 0.951158] pci_bus 0000:00: resource 7 [mem 0xc0000000-0xfebfffff window] [ 0.954048] pci_bus 0000:00: resource 8 [mem 0x380000000000-0x38007fffffff window] [ 0.957200] NET: Registered protocol family 2 [ 0.960323] IP idents hash table entries: 131072 (order: 8, 1048576 bytes, vmalloc) [ 0.966183] tcp_listen_portaddr_hash hash table entries: 4096 (order: 5, 163840 bytes, vmalloc) [ 0.969762] TCP established hash table entries: 65536 (order: 7, 524288 bytes, vmalloc) [ 0.976325] TCP bind hash table entries: 65536 (order: 9, 2097152 bytes, vmalloc) [ 0.979639] TCP: Hash tables configured (established 65536 bind 65536) [ 0.982727] MPTCP token hash table entries: 8192 (order: 6, 393216 bytes, vmalloc) [ 0.985865] UDP hash table entries: 4096 (order: 6, 393216 bytes, vmalloc) [ 0.988949] UDP-Lite hash table entries: 4096 (order: 6, 393216 bytes, vmalloc) [ 0.992199] NET: Registered protocol family 1 [ 0.995421] RPC: Registered named UNIX socket transport module. [ 0.998241] RPC: Registered udp transport module. [ 1.000015] RPC: Registered tcp transport module. [ 1.001784] RPC: Registered tcp NFSv4.1 backchannel transport module. [ 1.004353] NET: Registered protocol family 44 [ 1.006088] pci 0000:00:00.0: Limiting direct PCI/PCI transfers [ 1.008374] pci 0000:00:01.0: PIIX3: Enabling Passive Release [ 1.010669] pci 0000:00:01.0: Activating ISA DMA hang workarounds [ 1.013572] PCI: CLS 0 bytes, default 64 [ 1.016295] Unpacking initramfs... [ 2.505867] debug: unmapping init [mem 0xffff9e04bcc54000-0xffff9e04bffbffff] [ 2.512747] PCI-DMA: Using software bounce buffering for IO (SWIOTLB) [ 2.515601] software IO TLB: mapped [mem 0x00000000a8000000-0x00000000ac000000] (64MB) [ 2.519234] clocksource: tsc: mask: 0xffffffffffffffff max_cycles: 0x229835b7123, max_idle_ns: 440795242976 ns [ 3.035170] Initialise system trusted keyrings [ 3.037072] Key type blacklist registered [ 3.039157] workingset: timestamp_bits=36 max_order=20 bucket_order=0 [ 3.050788] zbud: loaded [ 3.054052] *** VALIDATE nfs *** [ 3.055227] *** VALIDATE nfs4 *** [ 3.058332] pstore: using deflate compression [ 3.064876] Platform Keyring initialized [ 3.188113] NET: Registered protocol family 38 [ 3.192144] Key type asymmetric registered [ 3.193564] Asymmetric key parser 'x509' registered [ 3.195552] Block layer SCSI generic (bsg) driver version 0.4 loaded (major 247) [ 3.199195] io scheduler mq-deadline registered [ 3.200668] io scheduler kyber registered [ 3.202330] io scheduler bfq registered [ 3.204454] atomic64_test: passed for x86-64 platform with CX8 and with SSE [ 3.208286] shpchp: Standard Hot Plug PCI Controller Driver version: 0.4 [ 3.211506] input: Power Button as /devices/LNXSYSTM:00/LNXPWRBN:00/input/input0 [ 3.214826] ACPI: Power Button [PWRF] [ 3.308433] ACPI: \_SB_.LNKB: Enabled at IRQ 10 [ 3.401103] ACPI: \_SB_.LNKA: Enabled at IRQ 11 [ 3.584900] ACPI: \_SB_.LNKC: Enabled at IRQ 11 [ 3.679405] ACPI: \_SB_.LNKD: Enabled at IRQ 10 [ 3.871195] Serial: 8250/16550 driver, 4 ports, IRQ sharing enabled [ 3.900502] 00:03: ttyS1 at I/O 0x2f8 (irq = 3, base_baud = 115200) is a 16550A [ 3.931483] 00:04: ttyS0 at I/O 0x3f8 (irq = 4, base_baud = 115200) is a 16550A [ 3.936654] Non-volatile memory driver v1.3 [ 3.938800] Linux agpgart interface v0.103 [ 3.973414] virtio_blk virtio1: [vda] 133840 512-byte logical blocks (68.5 MB/65.4 MiB) [ 3.977930] vda: detected capacity change from 0 to 68526080 [ 3.997484] virtio_blk virtio2: [vdb] 2097152 512-byte logical blocks (1.07 GB/1.00 GiB) [ 4.001354] vdb: detected capacity change from 0 to 1073741824 [ 4.020421] virtio_blk virtio3: [vdc] 5120000 512-byte logical blocks (2.62 GB/2.44 GiB) [ 4.023216] vdc: detected capacity change from 0 to 2621440000 [ 4.041761] virtio_blk virtio4: [vdd] 5120000 512-byte logical blocks (2.62 GB/2.44 GiB) [ 4.044422] vdd: detected capacity change from 0 to 2621440000 [ 4.068954] virtio_blk virtio5: [vde] 8388608 512-byte logical blocks (4.29 GB/4.00 GiB) [ 4.072335] vde: detected capacity change from 0 to 4294967296 [ 4.087887] virtio_blk virtio6: [vdf] 8388608 512-byte logical blocks (4.29 GB/4.00 GiB) [ 4.091017] vdf: detected capacity change from 0 to 4294967296 [ 4.098082] libphy: Fixed MDIO Bus: probed [ 4.105942] usbcore: registered new interface driver usbserial_generic [ 4.108298] usbserial: USB Serial support registered for generic [ 4.110486] i8042: PNP: PS/2 Controller [PNP0303:KBD,PNP0f13:MOU] at 0x60,0x64 irq 1,12 [ 4.114453] serio: i8042 KBD port at 0x60,0x64 irq 1 [ 4.116320] serio: i8042 AUX port at 0x60,0x64 irq 12 [ 4.118896] mousedev: PS/2 mouse device common for all mice [ 4.122246] rtc_cmos 00:05: RTC can wake from S4 [ 4.125235] rtc_cmos 00:05: registered as rtc0 [ 4.127491] rtc_cmos 00:05: alarms up to one day, y3k, 242 bytes nvram, hpet irqs [ 4.129076] input: AT Translated Set 2 keyboard as /devices/platform/i8042/serio0/input/input1 [ 4.131327] intel_pstate: CPU model not supported [ 4.137638] input: VirtualPS/2 VMware VMMouse as /devices/platform/i8042/serio1/input/input4 [ 4.138082] hid: raw HID events driver (C) Jiri Kosina [ 4.146070] usbcore: registered new interface driver usbhid [ 4.146272] input: VirtualPS/2 VMware VMMouse as /devices/platform/i8042/serio1/input/input3 [ 4.148662] usbhid: USB HID core driver [ 4.148873] drop_monitor: Initializing network drop monitor service [ 4.154837] Initializing XFRM netlink socket [ 4.156749] NET: Registered protocol family 10 [ 4.159630] Segment Routing with IPv6 [ 4.161439] NET: Registered protocol family 17 [ 4.163556] mpls_gso: MPLS GSO support [ 4.169717] RAS: Correctable Errors collector initialized. [ 4.172127] AVX version of gcm_enc/dec engaged. [ 4.173864] AES CTR mode by8 optimization enabled [ 4.272523] sched_clock: Marking stable (4272499309, 0)->(5215402442, -942903133) [ 4.275537] registered taskstats version 1 [ 4.277597] Loading compiled-in X.509 certificates [ 4.279597] zswap: loaded using pool lzo/zbud [ 4.305182] Key type big_key registered [ 4.316610] Key type encrypted registered [ 4.318353] ima: No TPM chip found, activating TPM-bypass! [ 4.320189] ima: Allocated hash algorithm: sha1 [ 4.321912] ima: No architecture policies found [ 4.324686] evm: Initialising EVM extended attributes: [ 4.326573] evm: security.selinux [ 4.327817] evm: security.ima [ 4.328980] evm: security.capability [ 4.330305] evm: HMAC attrs: 0x1 [ 4.332727] rtc_cmos 00:05: setting system clock to 2025-11-17 03:53:09 UTC (1763351589) [ 4.339245] debug: unmapping init [mem 0xffffffffb5203000-0xffffffffb53fffff] [ 4.342028] debug: unmapping init [mem 0xffffffffb3f82000-0xffffffffb4258fff] [ 4.349094] Write protecting the kernel read-only data: 28672k [ 4.352617] debug: unmapping init [mem 0xffffffffb2603000-0xffffffffb27fffff] [ 4.355384] debug: unmapping init [mem 0xffffffffb2f14000-0xffffffffb2ffffff] [ 4.393723] systemd[1]: systemd 239 (239-82.el8_10.5) running in system mode. (+PAM +AUDIT +SELINUX +IMA -APPARMOR +SMACK +SYSVINIT +UTMP +LIBCRYPTSETUP +GCRYPT +GNUTLS +ACL +XZ +LZ4 +SECCOMP +BLKID +ELFUTILS +KMOD +IDN2 -IDN +PCRE2 default-hierarchy=legacy) [ 4.401610] systemd[1]: Detected virtualization kvm. [ 4.403662] systemd[1]: Detected architecture x86-64. [ 4.405534] systemd[1]: Running in initial RAM disk. Welcome to Rocky Linux 8.10 (Green Obsidian) dracut-049-233.git20240115.el8 (Initramfs)! [ 4.432861] systemd[1]: No hostname configured. [ 4.434372] systemd[1]: Set hostname to . [ 4.436824] random: systemd: uninitialized urandom read (16 bytes read) [ 4.439225] systemd[1]: Initializing machine ID from random generator. [ 4.554436] random: systemd: uninitialized urandom read (16 bytes read) [ 4.557576] systemd[1]: Listening on Journal Socket. [ OK ] Listening on Journal Socket. [ 4.562158] random: systemd: uninitialized urandom read (16 bytes read) [ 4.565070] systemd[1]: Listening on udev Control Socket. [ OK ] Listening on udev Control Socket. [ 4.569615] systemd[1]: Reached target Slices. [ OK ] Reached target Slices. [ OK ] Reached target Swap. [ OK ] Reached target Timers. [ OK ] Reached target Initrd Root Device. Starting Create list of required st…ce nodes for the current kernel... Starting Setup Virtual Console... [ OK ] Listening on Journal Socket (/dev/log). Starting Journal Service... Starting Apply Kernel Variables... [ OK ] Reached target Local File Systems. Starting Create Volatile Files and Directories... [ OK ] Started Memstrack Anylazing Service. [ OK ] Listening on udev Kernel Socket. [ OK ] Started Dispatch Password Requests to Console Directory Watch. [ OK ] Reached target Paths. [ OK ] Reached target Local Encrypted Volumes. [ OK ] Reached target Sockets. [ OK ] Started Create list of required sta…vice nodes for the current kernel. [ OK ] Started Setup Virtual Console. [ OK ] Started Apply Kernel Variables. [ OK ] Started Create Volatile Files and Directories. Starting dracut cmdline hook... Starting Create Static Device Nodes in /dev... [ OK ] Started Journal Service. [ OK ] Started Create Static Device Nodes in /dev. [ OK ] Started dracut cmdline hook. Starting dracut pre-udev hook... [ 5.182324] device-mapper: uevent: version 1.0.3 [ 5.184539] device-mapper: ioctl: 4.46.0-ioctl (2022-02-22) initialised: dm-devel@redhat.com [ OK ] Started dracut pre-udev hook. Starting udev Kernel Device Manager... [ OK ] Started udev Kernel Device Manager. Starting dracut pre-trigger hook... [ OK ] Started dracut pre-trigger hook. Starting udev Coldplug all Devices... Mounting Kernel Configuration File System... [ OK ] Mounted Kernel Configuration File System. [ OK ] Started udev Coldplug all Devices. [ OK ] Reached target System Initialization. [ OK ] Reached target Basic System. [ OK ] Started Hardware RNG Entropy Gatherer Daemon. Starting dracut initqueue hook... [ 5.951413] virtio_net virtio0 ens2: renamed from eth0 [ 5.981176] random: fast init done [ 6.008781] scsi host0: ata_piix [ 6.036697] scsi host1: ata_piix [ 6.038270] ata1: PATA max MWDMA2 cmd 0x1f0 ctl 0x3f6 bmdma 0xc320 irq 14 [ 6.040584] ata2: PATA max MWDMA2 cmd 0x170 ctl 0x376 bmdma 0xc328 irq 15 [ 9.941459] dracut-initqueue[580]: RTNETLINK answers: File exists [ 10.835631] random: crng init done [ 10.837262] random: 7 urandom warning(s) missed due to ratelimiting Starting nbd nbd0... [ OK ] Started nbd nbd0. [ OK ] Started dracut initqueue hook. [ OK ] Reached target Remote File Systems (Pre). [ OK ] Reached target Remote File Systems. Mounting /sysroot... [ 11.381739] EXT4-fs (nbd0): mounted filesystem with ordered data mode. Opts: (null) [ OK ] Mounted /sysroot. [ OK ] Reached target Initrd Root File System. Starting Reload Configuration from the Real Root... [ OK ] Started Reload Configuration from the Real Root. [ OK ] Reached target Initrd File Systems. [ OK ] Reached target Initrd Default Target. Starting dracut pre-pivot and cleanup hook... [ OK ] Started dracut pre-pivot and cleanup hook. Starting Cleaning Up and Shutting Down Daemons... Stopping Hardware RNG Entropy Gatherer Daemon... [ OK ] Stopped target Timers. [ OK ] Stopped dracut pre-pivot and cleanup hook. [ OK ] Stopped target Remote File Systems. [ OK ] Stopped target Remote File Systems (Pre). [ OK ] Stopped target Initrd Default Target. [ OK ] Stopped target Initrd Root Device. [ OK ] Stopped dracut initqueue hook. [ OK ] Stopped Hardware RNG Entropy Gatherer Daemon. [ OK ] Stopped target Basic System. [ OK ] Stopped target Sockets. [ OK ] Stopped target Slices. [ OK ] Stopped target Paths. [ OK ] Stopped target System Initialization. [ OK ] Stopped Create Volatile Files and Directories. [ OK ] Stopped target Local File Systems. [ OK ] Stopped target Local Encrypted Volumes. [ OK ] Stopped Dispatch Password Requests to Console Directory Watch. [ OK ] Stopped Apply Kernel Variables. [ OK ] Stopped udev Coldplug all Devices. [ OK ] Stopped dracut pre-trigger hook. Stopping udev Kernel Device Manager... [ OK ] Stopped target Swap. [ OK ] Started Cleaning Up and Shutting Down Daemons. [ OK ] Stopped udev Kernel Device Manager. [ OK ] Stopped dracut pre-udev hook. [ OK ] Stopped dracut cmdline hook. [ OK ] Stopped Create Static Device Nodes in /dev. [ OK ] Stopped Create list of required sta…vice nodes for the current kernel. [ OK ] Closed udev Kernel Socket. [ OK ] Closed udev Control Socket. Starting Cleanup udevd DB... [ OK ] Started Cleanup udevd DB. [ OK ] Reached target Switch Root. Starting Switch Root... [ 12.527491] printk: systemd: 24 output lines suppressed due to ratelimiting [ 12.784078] SELinux: Disabled at runtime. [ 12.847316] systemd[1]: systemd 239 (239-82.el8_10.5) running in system mode. (+PAM +AUDIT +SELINUX +IMA -APPARMOR +SMACK +SYSVINIT +UTMP +LIBCRYPTSETUP +GCRYPT +GNUTLS +ACL +XZ +LZ4 +SECCOMP +BLKID +ELFUTILS +KMOD +IDN2 -IDN +PCRE2 default-hierarchy=legacy) [ 12.855806] systemd[1]: Detected virtualization kvm. [ 12.857829] systemd[1]: Detected architecture x86-64. Welcome to Rocky Linux 8.10 (Green Obsidian)! [ 13.348300] systemd[1]: initrd-switch-root.service: Succeeded. [ 13.352106] systemd[1]: Stopped Switch Root. [ OK ] Stopped Switch Root. [ 13.356981] systemd[1]: systemd-journald.service: Service has no hold-off time (RestartSec=0), scheduling restart. [ 13.359907] systemd[1]: systemd-journald.service: Scheduled restart job, restart counter is at 1. [ 13.362181] systemd[1]: Stopped Journal Service. [ OK ] Stopped Journal Service. [ 13.368799] systemd[1]: Starting Journal Service... Starting Journal Service... [ 13.378205] systemd[1]: Mounting POSIX Message Queue File System... Mounting POSIX Message Queue File System... [ OK ] Listening on udev Kernel Socket. [ OK ] Created slice User and Session Slice. Starting Create list of required st…ce nodes for the current kernel... [ OK ] Created slice system-sshd\x2dkeygen.slice. Activating swap /dev/disk/by-label/SWAP... [ 13.448948] Adding 1048572k swap on /dev/vdb. Priority:-2 extents:1 across:1048572k FS [ OK ] Listening on Process Core Dump Socket. [ OK ] Reached target Slices. [ OK ] Reached target rpc_pipefs.target. Mounting Kernel Debug File System... [ OK ] Created slice system-serial\x2dgetty.slice. [ OK ] Created slice system-getty.slice. [ OK ] Started Forward Password Requests to Wall Directory Watch. [ OK ] Listening on udev Control Socket. Starting udev Coldplug all Devices... [ OK ] Listening on RPCbind Server Activation Socket. [ OK ] Reached target RPC Port Mapper. [ OK ] Stopped target Switch Root. [ OK ] Stopped target Initrd Root File System. Starting Remount Root and Kernel File Systems... [ OK ] Listening on initctl Compatibility Named Pipe. [ OK ] Started Dispatch Password Requests to Console Directory Watch. [ OK ] Reached target Local Encrypted Volumes. [ OK ] Reached target Paths. Mounting Huge Pages File System... Starting Apply Kernel Variables... [FAILED] Failed to set up automount Arbitrar…rmats File System Automount Point. See 'systemctl status proc-sys-fs-binfmt_misc.automount' for details. [ OK ] Stopped target Initrd File Systems. [ OK ] Started Journal Service. [ OK ] Mounted POSIX Message Queue File System. [ OK ] Started Create list of required sta…vice nodes for the current kernel. [ OK ] Activated swap /dev/disk/by-label/SWAP. [ OK ] Mounted Kernel Debug File System. [FAILED] Failed to start Remount Root and Kernel File Systems. See 'systemctl status systemd-remount-fs.service' for details. [ OK ] Mounted Huge Pages File System. [ OK ] Started Apply Kernel Variables. Starting Configure read-only root support... [ OK ] Reached target Swap. Starting Create Static Device Nodes in /dev... Starting Flush Journal to Persistent Storage... [ OK ] Started Flush Journal to Persistent Storage. [ OK ] Started Create Static Device Nodes in /dev. [ OK ] Reached target Local File Systems (Pre). Mounting /mnt... Mounting /home/green/git/lustre-release... Starting udev Kernel Device Manager... [ OK ] Started udev Coldplug all Devices. [ OK ] Mounted /mnt. [ 13.870250] squashfs: version 4.0 (2009/01/31) Phillip Lougher [ OK ] Mounted /home/green/git/lustre-release. [ OK ] Started udev Kernel Device Manager. [ 14.193940] piix4_smbus 0000:00:01.3: SMBus Host Controller at 0x700, revision 0 [ 14.253402] input: PC Speaker as /devices/platform/pcspkr/input/input5 [ 14.433671] RAPL PMU: API unit is 2^-32 Joules, 0 fixed counters, 10737418240 ms ovfl timer [ 14.444489] EDAC sbridge: Ver: 1.1.2 [ 16.268206] Key type dns_resolver registered [ 16.571525] NFS: Registering the id_resolver key type [ 16.573686] Key type id_resolver registered [ 16.575439] Key type id_legacy registered [ OK ] Started Configure read-only root support. [ OK ] Reached target Local File Systems. Starting Mark the need to relabel after reboot... Starting Rebuild Dynamic Linker Cache... Starting Create Volatile Files and Directories... Starting Load/Save Random Seed... [ OK ] Started Mark the need to relabel after reboot. [ OK ] Started Load/Save Random Seed. [ OK ] Started Create Volatile Files and Directories. Starting RPC Bind... Starting Update UTMP about System Boot/Shutdown... [ OK ] Started Update UTMP about System Boot/Shutdown. [ OK ] Started RPC Bind. [ OK ] Started Rebuild Dynamic Linker Cache. Starting Update is Completed... [ OK ] Started Update is Completed. [ OK ] Reached target System Initialization. [ OK ] Listening on D-Bus System Message Bus Socket. [ OK ] Reached target Sockets. [ OK ] Started Daily Cleanup of Temporary Directories. [ OK ] Started dnf makecache --timer. [ OK ] Reached target Basic System. Starting Restore /run/initramfs on shutdown... Starting Login Service... [ OK ] Started Hardware RNG Entropy Gatherer Daemon. [ OK ] Started irqbalance daemon. [ OK ] Reached target sshd-keygen.target. [ OK ] Started D-Bus System Message Bus. Starting Network Manager... [ OK ] Started daily update of the root trust anchor for DNSSEC. [ OK ] Reached target Timers. [ OK ] Started Restore /run/initramfs on shutdown. [ OK ] Started Login Service. [ OK ] Started Network Manager. Starting Network Manager Wait Online... [ OK ] Reached target Network. Starting GSSAPI Proxy Daemon... Starting OpenSSH server daemon... Starting Dynamic System Tuning Daemon... Starting Hostname Service... [ OK ] Started OpenSSH server daemon. [ OK ] Started GSSAPI Proxy Daemon. [ OK ] Reached target NFS client services. [ OK ] Reached target Remote File Systems (Pre). [ OK ] Reached target Remote File Systems. Starting Permit User Sessions... [ OK ] Started Permit User Sessions. [ OK ] Started Serial Getty on ttyS0. [ OK ] Started Command Scheduler. [ OK ] Started Getty on tty1. [ OK ] Started Serial Getty on ttyS1. [ OK ] Reached target Login Prompts. [ OK ] Started Hostname Service. Starting Network Manager Script Dispatcher Service... [ OK ] Started Network Manager Script Dispatcher Service. [ OK ] Started Network Manager Wait Online. [ OK ] Reached target Network is Online. Starting System Logging Service... Starting Notify NFS peers of a restart... Starting Crash recovery kernel arming... [ OK ] Started Notify NFS peers of a restart. [ OK ] Started System Logging Service. Starting Authorization Manager... [ OK ] Started Authorization Manager. [ OK ] Started Dynamic System Tuning Daemon. [ OK ] Reached target Multi-User System. [ OK ] Reached target Graphical Interface. Starting Update UTMP about System Runlevel Changes... [ OK ] Started Update UTMP about System Runlevel Changes. Rocky Linux 8.10 (Green Obsidian) Kernel 4.18.0rh8.10-debug on an x86_64 oleg140-server login: [ 42.918111] libcfs: loading out-of-tree module taints kernel. [ 42.933428] Key type ._llcrypt registered [ 42.934842] Key type .llcrypt registered [ 42.988318] Lustre: DEBUG MARKER: oleg140-server.virtnet: executing set_hostid [ 51.963350] Lustre: DEBUG MARKER: oleg140-server.virtnet: executing load_modules_local [ 52.754184] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 52.762625] alg: No test for adler32 (adler32-zlib) [ 53.817218] Lustre: Lustre: Build Version: 2.16.61_41_g6a6a76f [ 54.180142] LNet: Added LNI 192.168.201.140@tcp [8/256/0/180] [ 55.810210] Key type lgssc registered [ 57.461623] Lustre: Echo OBD driver; http://www.lustre.org/ [ 60.470384] hrtimer: interrupt took 3366088 ns [ 72.298955] ZFS: Loaded module v2.3.2-1, ZFS pool version 5000, ZFS filesystem version 5 [ 73.864286] LDISKFS-fs (vdc): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 80.628265] LDISKFS-fs (vdd): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 87.242897] LDISKFS-fs (vde): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 93.150834] LDISKFS-fs (vdf): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 105.062574] Lustre: DEBUG MARKER: oleg140-server.virtnet: executing load_modules_local [ 115.824502] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 115.875937] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 115.891763] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 117.088380] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 117.109970] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 117.212739] Lustre: lustre-MDT0000: new disk, initializing [ 117.310409] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 117.329281] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 120.509366] Lustre: DEBUG MARKER: oleg140-server.virtnet: executing set_default_debug vfstrace rpctrace dlmtrace neterror ha config ioctl super lfsck all [ 131.354586] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 131.407675] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 131.458867] Lustre: 6482:0:(mgs_llog.c:1346:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 131.485230] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [ 131.487440] Lustre: Skipped 1 previous similar message [ 131.549349] Lustre: lustre-MDT0001: new disk, initializing [ 131.616923] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 131.649876] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [ 131.653911] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 134.722268] Lustre: DEBUG MARKER: oleg140-server.virtnet: executing set_default_debug vfstrace rpctrace dlmtrace neterror ha config ioctl super lfsck all [ 138.752388] Lustre: Modifying parameter general.debug_raw_pointers=Y in log params [ 147.193292] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 147.251778] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 147.468749] Lustre: lustre-OST0000: new disk, initializing [ 147.474713] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 147.560158] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 148.838792] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [ 148.847823] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 148.930728] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 151.881148] Lustre: DEBUG MARKER: oleg140-server.virtnet: executing set_default_debug vfstrace rpctrace dlmtrace neterror ha config ioctl super lfsck all [ 163.089842] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 163.156616] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 163.224502] Lustre: lustre-OST0001: new disk, initializing [ 163.227899] Lustre: srv-lustre-OST0001: No data found on store. Initialize space. [ 163.272349] Lustre: lustre-OST0001: Imperative Recovery not enabled, recovery window 60-180 [ 166.903073] Lustre: DEBUG MARKER: oleg140-server.virtnet: executing set_default_debug vfstrace rpctrace dlmtrace neterror ha config ioctl super lfsck all [ 172.570235] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x00000002c0000400-0x0000000300000400]:1:ost [ 172.577405] Lustre: cli-lustre-OST0001-super: Allocated super-sequence [0x00000002c0000400-0x0000000300000400]:1:ost] [ 172.628681] Lustre: lustre-OST0001-osc-MDT0000: update sequence from 0x100010000 to 0x2c0000401 [ 176.152660] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 181.579216] Lustre: Setting parameter general.lod.*.mdt_hash=crush in log params [ 191.179253] Lustre: DEBUG MARKER: oleg140-server.virtnet: executing check_logdir /tmp/testlogs/ [ 194.630392] Lustre: DEBUG MARKER: oleg140-server.virtnet: executing yml_node [ 198.242059] Lustre: DEBUG MARKER: Client: 2.16.61.41 [ 200.449789] Lustre: DEBUG MARKER: MDS: 2.16.61.41 [ 202.570791] Lustre: DEBUG MARKER: OSS: 2.16.61.41 [ 203.928350] Lustre: DEBUG MARKER: -----============= acceptance-small: conf-sanity ============----- Sun Nov 16 22:56:27 EST 2025 [ 217.772164] Lustre: DEBUG MARKER: excepting tests: 102 106 115 32newtarball [ 219.022802] Lustre: DEBUG MARKER: skipping tests SLOW=no: 45 69 106 111 114 [ 233.954535] Lustre: lustre-MDT0000-lwp-OST0000: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 233.955639] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 233.957521] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 233.962544] Lustre: Skipped 1 previous similar message [ 239.073388] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 239.078898] Lustre: Skipped 6 previous similar messages [ 244.195206] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 247.264946] Lustre: lustre-MDT0000 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 247.331418] LustreError: 12185:0:(obd_class.h:479:obd_check_dev()) Device 28 not setup [ 247.391990] Lustre: server umount lustre-MDT0000 complete [ 249.788357] LustreError: 6473:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1763351834 with bad export cookie 10142267141875050196 [ 249.794106] LustreError: MGC192.168.201.140@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 249.795635] LustreError: 6473:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 254.433341] LustreError: 11205:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 254.439918] Lustre: lustre-MDT0001-lwp-OST0000: Connection to lustre-MDT0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 254.445843] LustreError: 11205:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 6 previous similar messages [ 254.451690] Lustre: Skipped 1 previous similar message [ 254.458218] Lustre: lustre-MDT0001: Not available for connect from 0@lo (stopping) [ 254.462627] Lustre: Skipped 4 previous similar messages [ 259.553167] LustreError: 11205:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 259.553849] Lustre: lustre-MDT0001: Not available for connect from 0@lo (stopping) [ 259.573621] LustreError: 11205:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 1 previous similar message [ 259.584912] Lustre: Skipped 1 previous similar message [ 260.577713] LustreError: 6489:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 260.585168] LustreError: 6489:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 1 previous similar message [ 264.160139] Lustre: lustre-MDT0001 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 264.240844] LustreError: 12388:0:(obd_class.h:479:obd_check_dev()) Device 27 not setup [ 264.245646] LustreError: 12388:0:(obd_class.h:479:obd_check_dev()) Skipped 7 previous similar messages [ 264.359400] Lustre: server umount lustre-MDT0001 complete [ 267.129317] LustreError: 12591:0:(obd_class.h:479:obd_check_dev()) Device 19 not setup [ 267.136992] LustreError: 12591:0:(obd_class.h:479:obd_check_dev()) Skipped 7 previous similar messages [ 267.170704] Lustre: server umount lustre-OST0000 complete [ 269.682235] LustreError: 12791:0:(obd_class.h:479:obd_check_dev()) Device 2 not setup [ 269.688543] LustreError: 12791:0:(obd_class.h:479:obd_check_dev()) Skipped 3 previous similar messages [ 269.783140] Lustre: server umount lustre-OST0001 complete [ 274.088591] Lustre: DEBUG MARKER: oleg140-server.virtnet: executing set_hostid [ 280.330644] Lustre: DEBUG MARKER: oleg140-server.virtnet: executing load_modules_local [ 287.002048] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 291.126842] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 295.266435] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 300.331606] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 305.188327] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 305.246429] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 305.423700] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 305.464268] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 305.543702] Lustre: lustre-MDT0000: new disk, initializing [ 305.615904] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 305.633883] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 308.330343] Lustre: DEBUG MARKER: oleg140-server.virtnet: executing set_default_debug -1 all [ 315.910755] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 315.960233] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 316.032780] Lustre: 14972:0:(mgs_llog.c:1346:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 316.077257] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [ 316.079733] Lustre: Skipped 1 previous similar message [ 316.133226] Lustre: lustre-MDT0001: new disk, initializing [ 316.210621] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 316.237201] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [ 316.250839] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 318.757984] Lustre: DEBUG MARKER: oleg140-server.virtnet: executing set_default_debug -1 all [ 323.649373] Lustre: DEBUG MARKER: oleg140-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 325.811600] Lustre: DEBUG MARKER: oleg140-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 330.358657] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 330.397418] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 331.741157] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 331.844666] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 334.305149] Lustre: DEBUG MARKER: oleg140-server.virtnet: executing set_default_debug -1 all [ 339.548201] Lustre: DEBUG MARKER: oleg140-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 341.981423] Lustre: DEBUG MARKER: oleg140-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 [ 342.124110] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [ 344.430806] Lustre: DEBUG MARKER: oleg140-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid 50 [ 344.558521] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [ 347.108270] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 347.108664] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 347.118627] LustreError: Skipped 1 previous similar message [ 347.123249] Lustre: Skipped 1 previous similar message [ 347.125548] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 347.133475] Lustre: Skipped 3 previous similar messages [ 351.671241] LustreError: 17914:0:(obd_class.h:479:obd_check_dev()) Device 19 not setup [ 351.674483] LustreError: 17914:0:(obd_class.h:479:obd_check_dev()) Skipped 5 previous similar messages [ 351.749720] Lustre: server umount lustre-OST0000 complete [ 357.346420] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 357.347268] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 357.351449] Lustre: Skipped 1 previous similar message [ 360.087294] LustreError: 18115:0:(obd_class.h:479:obd_check_dev()) Device 22 not setup [ 360.089862] LustreError: 18115:0:(obd_class.h:479:obd_check_dev()) Skipped 3 previous similar messages [ 360.154165] Lustre: server umount lustre-MDT0000 complete [ 362.465342] LustreError: 14983:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 362.475221] LustreError: 14983:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 1 previous similar message [ 362.490695] LustreError: 14964:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1763351947 with bad export cookie 10142267141875051729 [ 362.494539] LustreError: MGC192.168.201.140@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 362.497100] LustreError: 14964:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 362.670199] Lustre: server umount lustre-MDT0001 complete [ 366.012757] Lustre: DEBUG MARKER: == conf-sanity test 76a: set permanent params with lctl across mounts ========================================================== 22:59:10 (1763351950) [ 370.212669] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 370.458738] LustreError: 18841:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 370.504498] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 370.511456] Lustre: Skipped 1 previous similar message [ 372.656468] Lustre: DEBUG MARKER: oleg140-server.virtnet: executing set_default_debug -1 all [ 377.335950] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 379.655974] Lustre: DEBUG MARKER: oleg140-server.virtnet: executing set_default_debug -1 all [ 382.878171] Lustre: DEBUG MARKER: oleg140-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 384.896110] Lustre: DEBUG MARKER: oleg140-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 389.213871] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 392.801504] Lustre: DEBUG MARKER: oleg140-server.virtnet: executing set_default_debug -1 all [ 395.806237] Lustre: DEBUG MARKER: oleg140-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 407.194396] Lustre: Modifying parameter general.osc.*.max_dirty_mb=452 in log params [ 413.156836] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 413.161551] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 413.171290] Lustre: Skipped 2 previous similar messages [ 413.176194] Lustre: Skipped 4 previous similar messages [ 413.469806] LustreError: 21162:0:(obd_class.h:479:obd_check_dev()) Device 9 not setup [ 413.473023] LustreError: 21162:0:(obd_class.h:479:obd_check_dev()) Skipped 13 previous similar messages [ 413.525481] Lustre: server umount lustre-MDT0000 complete [ 415.936848] LustreError: 18821:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1763352001 with bad export cookie 10142267141875052611 [ 415.945823] LustreError: MGC192.168.201.140@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 415.957178] LustreError: 18821:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 416.163657] Lustre: server umount lustre-MDT0001 complete [ 428.873203] Lustre: server umount lustre-OST0000 complete [ 440.717238] Lustre: DEBUG MARKER: oleg140-server.virtnet: executing load_modules_local [ 446.394270] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 446.637250] LustreError: 22891:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 446.645815] LustreError: 22891:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 1 previous similar message [ 446.676439] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 446.679502] Lustre: Skipped 2 previous similar messages [ 449.216539] Lustre: DEBUG MARKER: oleg140-server.virtnet: executing set_default_debug -1 all [ 454.456578] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 457.276768] Lustre: DEBUG MARKER: oleg140-server.virtnet: executing set_default_debug -1 all [ 459.156630] Lustre: Setting parameter general.debug_raw_pointers=Y in log params [ 463.238570] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 467.061405] Lustre: DEBUG MARKER: oleg140-server.virtnet: executing set_default_debug -1 all [ 472.653957] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 472.701771] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 472.802301] Lustre: lustre-OST0001: new disk, initializing [ 472.804570] Lustre: Skipped 1 previous similar message [ 472.808488] Lustre: srv-lustre-OST0001: No data found on store. Initialize space. [ 472.815119] Lustre: Skipped 1 previous similar message [ 474.162259] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x00000002c0000400-0x0000000300000400]:1:ost [ 474.169810] Lustre: Skipped 1 previous similar message [ 474.171997] Lustre: cli-lustre-OST0001-super: Allocated super-sequence [0x00000002c0000400-0x0000000300000400]:1:ost] [ 474.230630] Lustre: lustre-OST0001-osc-MDT0000: update sequence from 0x100010000 to 0x2c0000401 [ 476.363205] Lustre: DEBUG MARKER: oleg140-server.virtnet: executing set_default_debug -1 all [ 483.081647] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 485.334327] Lustre: Setting parameter general.lod.*.mdt_hash=crush in log params [ 499.550854] LustreError: 26387:0:(obd_class.h:479:obd_check_dev()) Device 28 not setup [ 499.553870] LustreError: 26387:0:(obd_class.h:479:obd_check_dev()) Skipped 17 previous similar messages [ 499.596698] Lustre: server umount lustre-MDT0000 complete [ 499.686783] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 499.693448] LustreError: 23604:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 499.695419] Lustre: Skipped 2 previous similar messages [ 499.713250] LustreError: 23604:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 4 previous similar messages [ 501.744864] LustreError: 25162:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1763352086 with bad export cookie 10142267141875053458 [ 501.749569] LustreError: MGC192.168.201.140@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 501.751542] LustreError: 25162:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 501.933238] Lustre: server umount lustre-MDT0001 complete [ 514.092238] Lustre: server umount lustre-OST0000 complete [ 526.539573] Lustre: server umount lustre-OST0001 complete [ 535.661083] Lustre: DEBUG MARKER: oleg140-server.virtnet: executing load_modules_local [ 541.758466] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 541.989585] LustreError: 28163:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 542.054217] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 542.058647] Lustre: Skipped 3 previous similar messages [ 544.538336] Lustre: DEBUG MARKER: oleg140-server.virtnet: executing set_default_debug -1 all [ 549.962280] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 552.580210] Lustre: DEBUG MARKER: oleg140-server.virtnet: executing set_default_debug -1 all [ 554.711602] Lustre: 29278:0:(mgs_llog.c:1346:mgs_modify_param()) MGS: modify general/debug_raw_pointers=Y (mode = 0) failed: rc = -17 [ 559.412820] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 563.402136] Lustre: DEBUG MARKER: oleg140-server.virtnet: executing set_default_debug -1 all [ 569.130127] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 572.661686] Lustre: DEBUG MARKER: oleg140-server.virtnet: executing set_default_debug -1 all [ 577.651062] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 579.927574] Lustre: 31132:0:(mgs_llog.c:1346:mgs_modify_param()) MGS: modify general/lod.*.mdt_hash=crush (mode = 0) failed: rc = -17 [ 589.797075] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 589.800316] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 589.803475] Lustre: Skipped 5 previous similar messages [ 589.806473] Lustre: Skipped 2 previous similar messages [ 594.769084] LustreError: 31335:0:(obd_class.h:479:obd_check_dev()) Device 10 not setup [ 594.772196] LustreError: 31335:0:(obd_class.h:479:obd_check_dev()) Skipped 25 previous similar messages [ 594.817856] Lustre: server umount lustre-MDT0000 complete [ 596.992264] LustreError: 28142:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1763352182 with bad export cookie 10142267141875054753 [ 596.998524] LustreError: MGC192.168.201.140@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 597.004876] LustreError: 28142:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 597.197682] Lustre: server umount lustre-MDT0001 complete [ 609.301617] Lustre: server umount lustre-OST0000 complete [ 621.829356] Lustre: server umount lustre-OST0001 complete [ 626.905476] Lustre: DEBUG MARKER: == conf-sanity test 76b: verify params log setup correctly ========================================================== 23:03:31 (1763352211) [ 641.952863] Lustre: DEBUG MARKER: oleg140-server.virtnet: executing load_modules_local [ 648.487037] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 648.747307] LustreError: 33916:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 648.775829] LustreError: 33916:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 10 previous similar messages [ 651.313763] Lustre: DEBUG MARKER: oleg140-server.virtnet: executing set_default_debug -1 all [ 656.838209] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 659.611740] Lustre: DEBUG MARKER: oleg140-server.virtnet: executing set_default_debug -1 all [ 661.531652] Lustre: 35030:0:(mgs_llog.c:1346:mgs_modify_param()) MGS: modify general/debug_raw_pointers=Y (mode = 0) failed: rc = -17 [ 665.416603] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 669.012631] Lustre: DEBUG MARKER: oleg140-server.virtnet: executing set_default_debug -1 all [ 674.735778] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 674.902919] Lustre: lustre-OST0001: Imperative Recovery not enabled, recovery window 60-180 [ 674.910653] Lustre: Skipped 6 previous similar messages [ 678.616835] Lustre: DEBUG MARKER: oleg140-server.virtnet: executing set_default_debug -1 all [ 683.473444] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 685.707543] Lustre: 36888:0:(mgs_llog.c:1346:mgs_modify_param()) MGS: modify general/lod.*.mdt_hash=crush (mode = 0) failed: rc = -17 [ 693.217814] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 693.222925] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 693.232395] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 699.155970] Lustre: server umount lustre-MDT0000 complete [ 701.298691] LustreError: 33897:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1763352286 with bad export cookie 10142267141875055859 [ 701.302091] LustreError: MGC192.168.201.140@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 701.304513] LustreError: 33897:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 701.506701] Lustre: server umount lustre-MDT0001 complete [ 714.773612] Lustre: server umount lustre-OST0000 complete [ 726.853557] LustreError: 37644:0:(obd_class.h:479:obd_check_dev()) Device 2 not setup [ 726.857714] LustreError: 37644:0:(obd_class.h:479:obd_check_dev()) Skipped 45 previous similar messages [ 726.929397] Lustre: server umount lustre-OST0001 complete [ 731.908466] Lustre: DEBUG MARKER: == conf-sanity test 76c: verify changelog_mask is applied with lctl set_param -P ========================================================== 23:05:16 (1763352316) [ 740.249121] Lustre: DEBUG MARKER: oleg140-server.virtnet: executing load_modules_local [ 745.772654] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 748.268790] Lustre: DEBUG MARKER: oleg140-server.virtnet: executing set_default_debug -1 all [ 753.762781] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 756.477824] Lustre: DEBUG MARKER: oleg140-server.virtnet: executing set_default_debug -1 all [ 758.250516] Lustre: 40127:0:(mgs_llog.c:1346:mgs_modify_param()) MGS: modify general/debug_raw_pointers=Y (mode = 0) failed: rc = -17 [ 762.043423] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 765.184967] Lustre: DEBUG MARKER: oleg140-server.virtnet: executing set_default_debug -1 all [ 770.115700] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 773.964368] Lustre: DEBUG MARKER: oleg140-server.virtnet: executing set_default_debug -1 all [ 778.761642] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 786.235947] Lustre: Setting parameter general.mdd.*.changelog_mask=-CLOSE in log params [ 786.239840] Lustre: Skipped 1 previous similar message [ 789.985910] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 789.989682] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 789.995714] Lustre: Skipped 3 previous similar messages [ 794.128659] Lustre: server umount lustre-MDT0000 complete [ 796.098219] LustreError: 39006:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0000: not available for connect from 192.168.201.40@tcp (no target). If you are running an HA pair check that the target is mounted on the other server. [ 796.106103] LustreError: 39006:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 13 previous similar messages [ 796.456401] LustreError: 40510:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1763352381 with bad export cookie 10142267141875056965 [ 796.459245] LustreError: MGC192.168.201.140@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 796.465385] LustreError: 40510:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 802.751091] Lustre: server umount lustre-MDT0001 complete [ 807.563447] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 821.735050] LustreError: 3622:0:(client.c:1390:ptlrpc_import_delay_req()) @@@ invalidate in flight req@ffff9e0404405500 x1849008208573952/t0(0) o250->MGC192.168.201.140@tcp@0@lo:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl Rpc:NQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 [ 824.265941] Lustre: DEBUG MARKER: oleg140-server.virtnet: executing set_default_debug -1 all [ 829.661701] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 830.957285] Lustre: lustre-MDT0000-lwp-OST0000: Connection restored to 0@lo (at 0@lo) [ 832.214562] Lustre: DEBUG MARKER: oleg140-server.virtnet: executing set_default_debug -1 all [ 835.426446] Lustre: DEBUG MARKER: oleg140-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 836.529949] Lustre: DEBUG MARKER: mdc.lustre-MDT0000-mdc-*.mds_server_uuid in FULL state after 0 sec [ 838.845277] Lustre: DEBUG MARKER: oleg140-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 842.954865] Lustre: DEBUG MARKER: mdc.lustre-MDT0001-mdc-*.mds_server_uuid in FULL state after 3 sec [ 846.313319] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 846.316102] Lustre: Skipped 18 previous similar messages [ 851.787116] Lustre: server umount lustre-MDT0000 complete [ 853.828665] LustreError: 42040:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1763352438 with bad export cookie 10142267141875058141 [ 853.835338] LustreError: MGC192.168.201.140@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 854.054128] Lustre: server umount lustre-MDT0001 complete [ 866.781296] Lustre: server umount lustre-OST0000 complete [ 878.113423] Lustre: server umount lustre-OST0001 complete [ 883.387480] Lustre: DEBUG MARKER: == conf-sanity test 76d: verify llite.*.xattr_cache can be set by 'lctl set_param -P' correctly ========================================================== 23:07:47 (1763352467) [ 891.993207] Lustre: DEBUG MARKER: oleg140-server.virtnet: executing load_modules_local [ 897.494985] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 900.104355] Lustre: DEBUG MARKER: oleg140-server.virtnet: executing set_default_debug -1 all [ 905.061432] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 907.504666] Lustre: DEBUG MARKER: oleg140-server.virtnet: executing set_default_debug -1 all [ 909.120954] Lustre: 47471:0:(mgs_llog.c:1346:mgs_modify_param()) MGS: modify general/debug_raw_pointers=Y (mode = 0) failed: rc = -17 [ 909.125138] Lustre: 47471:0:(mgs_llog.c:1346:mgs_modify_param()) Skipped 1 previous similar message [ 912.877900] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 916.415611] Lustre: DEBUG MARKER: oleg140-server.virtnet: executing set_default_debug -1 all [ 921.412909] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 924.535375] Lustre: DEBUG MARKER: oleg140-server.virtnet: executing set_default_debug -1 all [ 929.196751] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 936.973430] Lustre: Setting parameter general.llite.*.xattr_cache=0 in log params [ 946.149277] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 946.152963] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 946.160533] Lustre: Skipped 9 previous similar messages [ 950.949711] Lustre: server umount lustre-MDT0000 complete [ 953.185215] LustreError: 46330:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1763352538 with bad export cookie 10142267141875058925 [ 953.188962] LustreError: MGC192.168.201.140@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 953.193122] LustreError: 46330:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) Skipped 6 previous similar messages [ 953.398597] Lustre: server umount lustre-MDT0001 complete [ 962.132821] Lustre: server umount lustre-OST0000 complete [ 964.663473] Lustre: server umount lustre-OST0001 complete [ 971.223828] Lustre: DEBUG MARKER: == conf-sanity test 77: comma-separated MGS NIDs and failover node NIDs ========================================================== 23:09:15 (1763352555) [ 972.664505] Lustre: DEBUG MARKER: SKIP: conf-sanity test_77 mixed loopback and real device not working [ 973.659317] Lustre: DEBUG MARKER: == conf-sanity test 78: run resize2fs on MDT and OST filesystems ========================================================== 23:09:17 (1763352557) [ 979.316939] Lustre: DEBUG MARKER: oleg140-server.virtnet: executing set_hostid [ 983.832759] Lustre: DEBUG MARKER: oleg140-server.virtnet: executing load_modules_local [ 988.322378] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 991.607852] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 995.382594] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 995.431270] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 995.585464] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 995.637460] Lustre: lustre-MDT0000: new disk, initializing [ 995.695862] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 995.699904] Lustre: Skipped 10 previous similar messages [ 995.711613] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 997.765079] Lustre: DEBUG MARKER: oleg140-server.virtnet: executing set_default_debug -1 all [ 1002.293366] Lustre: DEBUG MARKER: oleg140-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 1005.891895] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 1005.935292] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 1007.787943] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:0:ost] [ 1007.808630] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x240000400 [ 1009.050385] Lustre: DEBUG MARKER: oleg140-server.virtnet: executing set_default_debug -1 all [ 1013.120418] Lustre: DEBUG MARKER: oleg140-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 1014.231100] Lustre: DEBUG MARKER: create test files [ 1033.698301] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 1035.891497] LustreError: 54144:0:(obd_class.h:479:obd_check_dev()) Device 12 not setup [ 1035.894794] LustreError: 54144:0:(obd_class.h:479:obd_check_dev()) Skipped 73 previous similar messages [ 1036.022438] Lustre: server umount lustre-OST0000 complete [ 1038.247619] Lustre: server umount lustre-MDT0000 complete [ 1043.263660] Lustre: DEBUG MARKER: oleg140-server.virtnet: executing unload_modules_local [ 1044.632764] Key type lgssc unregistered [ 1044.803556] LNet: 54878:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 1044.808176] LNetError: 54878:0:(acceptor.c:264:lnet_acceptor_remove_socket()) Interface ens2 not found [ 1044.817656] LNet: Removed LNI 192.168.201.140@tcp [ 1045.297175] Key type .llcrypt unregistered [ 1045.298901] Key type ._llcrypt unregistered [ 1060.663096] Key type ._llcrypt registered [ 1060.664771] Key type .llcrypt registered [ 1060.729898] Lustre: DEBUG MARKER: oleg140-server.virtnet: executing load_modules_local [ 1061.303248] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 1061.377521] alg: No test for adler32 (adler32-zlib) [ 1062.356243] Lustre: Lustre: Build Version: 2.16.61_41_g6a6a76f [ 1062.533278] LNet: Added LNI 192.168.201.140@tcp [8/256/0/180] [ 1064.160224] Key type lgssc registered [ 1064.976827] Lustre: Echo OBD driver; http://www.lustre.org/ [ 1069.838459] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 1069.867388] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1071.168365] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 1073.300773] Lustre: DEBUG MARKER: oleg140-server.virtnet: executing set_default_debug -1 all [ 1076.354145] Lustre: DEBUG MARKER: oleg140-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 1080.250565] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 1080.447356] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 1081.451716] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x240000400:102 to 0x240000400:129) [ 1081.486962] Lustre: 57461:0:(osd_internal.h:1472:osd_trans_exec_op()) lustre-OST0000: opcode 8: before 515 < left 544, rollback = 9 [ 1081.490805] Lustre: 57461:0:(osd_handler.c:2082:osd_trans_dump_creds()) create: 32/128/0, destroy: 0/0/0 [ 1081.494264] Lustre: 57461:0:(osd_handler.c:2089:osd_trans_dump_creds()) attr_set: 0/0/0, xattr_set: 0/0/0 [ 1081.497600] Lustre: 57461:0:(osd_handler.c:2099:osd_trans_dump_creds()) write: 2/13/1, punch: 0/0/0, quota 1/3/0 [ 1081.501225] Lustre: 57461:0:(osd_handler.c:2106:osd_trans_dump_creds()) insert: 32/544/0, delete: 0/0/0 [ 1081.504538] Lustre: 57461:0:(osd_handler.c:2113:osd_trans_dump_creds()) ref_add: 0/0/0, ref_del: 0/0/0 [ 1083.054545] Lustre: DEBUG MARKER: oleg140-server.virtnet: executing set_default_debug -1 all [ 1086.084551] Lustre: DEBUG MARKER: oleg140-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 1091.077072] Lustre: DEBUG MARKER: check files after expanding the MDT and OST filesystems [ 1094.534485] Lustre: DEBUG MARKER: create more files after expanding the MDT and OST filesystems [ 1102.309365] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 1102.312102] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 1102.326433] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 1104.392182] LustreError: 58205:0:(obd_class.h:479:obd_check_dev()) Device 13 not setup [ 1104.497738] Lustre: server umount lustre-OST0000 complete [ 1107.135525] LustreError: 58406:0:(obd_class.h:479:obd_check_dev()) Device 8 not setup [ 1107.138335] LustreError: 58406:0:(obd_class.h:479:obd_check_dev()) Skipped 1 previous similar message [ 1107.244367] Lustre: server umount lustre-MDT0000 complete [ 1114.145379] Lustre: DEBUG MARKER: oleg140-server.virtnet: executing unload_modules_local [ 1115.733115] Key type lgssc unregistered [ 1115.970735] LNet: 58937:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 1115.975914] LNetError: 58937:0:(acceptor.c:264:lnet_acceptor_remove_socket()) Interface ens2 not found [ 1115.983893] LNet: Removed LNI 192.168.201.140@tcp [ 1116.486346] Key type .llcrypt unregistered [ 1116.488782] Key type ._llcrypt unregistered [ 1134.710874] Key type ._llcrypt registered [ 1134.712565] Key type .llcrypt registered [ 1134.766399] Lustre: DEBUG MARKER: oleg140-server.virtnet: executing load_modules_local [ 1135.311134] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 1135.324238] alg: No test for adler32 (adler32-zlib) [ 1136.245800] Lustre: Lustre: Build Version: 2.16.61_41_g6a6a76f [ 1136.375478] LNet: Added LNI 192.168.201.140@tcp [8/256/0/180] [ 1137.984281] Key type lgssc registered [ 1138.637049] Lustre: Echo OBD driver; http://www.lustre.org/ [ 1142.951579] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 1142.966163] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1144.248589] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 1146.159228] Lustre: DEBUG MARKER: oleg140-server.virtnet: executing set_default_debug -1 all [ 1148.891794] Lustre: DEBUG MARKER: oleg140-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 1152.180417] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 1152.347861] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 1154.416989] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x240000400:140 to 0x240000400:161) [ 1154.447175] Lustre: 61618:0:(osd_internal.h:1472:osd_trans_exec_op()) lustre-OST0000: opcode 8: before 515 < left 544, rollback = 9 [ 1154.450812] Lustre: 61618:0:(osd_handler.c:2082:osd_trans_dump_creds()) create: 32/128/0, destroy: 0/0/0 [ 1154.453945] Lustre: 61618:0:(osd_handler.c:2089:osd_trans_dump_creds()) attr_set: 0/0/0, xattr_set: 0/0/0 [ 1154.457523] Lustre: 61618:0:(osd_handler.c:2099:osd_trans_dump_creds()) write: 2/13/1, punch: 0/0/0, quota 1/3/0 [ 1154.461035] Lustre: 61618:0:(osd_handler.c:2106:osd_trans_dump_creds()) insert: 32/544/0, delete: 0/0/0 [ 1154.464161] Lustre: 61618:0:(osd_handler.c:2113:osd_trans_dump_creds()) ref_add: 0/0/0, ref_del: 0/0/0 [ 1155.127121] Lustre: DEBUG MARKER: oleg140-server.virtnet: executing set_default_debug -1 all [ 1158.028199] Lustre: DEBUG MARKER: oleg140-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 1162.804408] Lustre: DEBUG MARKER: check files after shrinking the MDT and OST filesystems [ 1169.890480] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 1169.891399] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 1169.913542] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 1174.200446] LustreError: 62312:0:(obd_class.h:479:obd_check_dev()) Device 13 not setup [ 1174.274376] Lustre: server umount lustre-OST0000 complete [ 1176.442604] LustreError: 62514:0:(obd_class.h:479:obd_check_dev()) Device 8 not setup [ 1176.445317] LustreError: 62514:0:(obd_class.h:479:obd_check_dev()) Skipped 1 previous similar message [ 1176.548526] Lustre: server umount lustre-MDT0000 complete [ 1182.282341] Lustre: DEBUG MARKER: oleg140-server.virtnet: executing unload_modules_local [ 1183.756214] Key type lgssc unregistered [ 1183.918243] LNet: 63045:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 1183.922831] LNetError: 63045:0:(acceptor.c:264:lnet_acceptor_remove_socket()) Interface ens2 not found [ 1183.933551] LNet: Removed LNI 192.168.201.140@tcp [ 1184.404273] Key type .llcrypt unregistered [ 1184.406750] Key type ._llcrypt unregistered [ 1194.673165] Key type ._llcrypt registered [ 1194.674716] Key type .llcrypt registered [ 1194.752689] Lustre: DEBUG MARKER: oleg140-server.virtnet: executing set_hostid [ 1203.506373] Lustre: DEBUG MARKER: oleg140-server.virtnet: executing load_modules_local [ 1203.990329] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 1204.059037] alg: No test for adler32 (adler32-zlib) [ 1204.973671] Lustre: Lustre: Build Version: 2.16.61_41_g6a6a76f [ 1205.109341] LNet: Added LNI 192.168.201.140@tcp [8/256/0/180] [ 1206.729732] Key type lgssc registered [ 1207.434756] Lustre: Echo OBD driver; http://www.lustre.org/ [ 1211.969476] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 1215.273076] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 1218.501839] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 1221.791581] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 1225.179635] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 1225.210225] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 1225.218022] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1226.342699] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 1226.362867] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 1226.410300] Lustre: lustre-MDT0000: new disk, initializing [ 1226.471771] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 1226.484603] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 1228.483574] Lustre: DEBUG MARKER: oleg140-server.virtnet: executing set_default_debug -1 all [ 1234.148024] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 1234.187501] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 1234.241559] Lustre: 66149:0:(mgs_llog.c:1346:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 1234.274308] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [ 1234.277141] Lustre: Skipped 1 previous similar message [ 1234.320644] Lustre: lustre-MDT0001: new disk, initializing [ 1234.373668] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 1234.392557] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [ 1234.397680] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 1236.285170] Lustre: DEBUG MARKER: oleg140-server.virtnet: executing set_default_debug -1 all [ 1240.831667] Lustre: DEBUG MARKER: oleg140-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 1242.740994] Lustre: DEBUG MARKER: oleg140-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 1246.754891] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 1246.793161] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 1246.955406] Lustre: lustre-OST0000: new disk, initializing [ 1246.958447] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 1247.018433] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 1250.637413] Lustre: DEBUG MARKER: oleg140-server.virtnet: executing set_default_debug -1 all [ 1255.465288] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [ 1255.476787] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 1255.525491] Lustre: DEBUG MARKER: oleg140-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 1255.541704] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 1257.831641] Lustre: DEBUG MARKER: oleg140-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 [ 1257.934644] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [ 1260.177507] Lustre: DEBUG MARKER: oleg140-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid 50 [ 1260.319950] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [ 1265.634180] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 1265.637949] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 1265.644431] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 1265.647610] LustreError: Skipped 1 previous similar message [ 1265.653157] Lustre: Skipped 1 previous similar message [ 1267.520534] LustreError: 69088:0:(obd_class.h:479:obd_check_dev()) Device 19 not setup [ 1267.602575] Lustre: server umount lustre-OST0000 complete [ 1270.242306] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 1270.246367] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 1270.256281] Lustre: Skipped 1 previous similar message [ 1270.259776] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 1275.789040] LustreError: 69290:0:(obd_class.h:479:obd_check_dev()) Device 22 not setup [ 1275.792216] LustreError: 69290:0:(obd_class.h:479:obd_check_dev()) Skipped 3 previous similar messages [ 1275.833437] Lustre: server umount lustre-MDT0000 complete [ 1277.729193] LustreError: 66141:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1763352862 with bad export cookie 4397838282544780765 [ 1277.729854] LustreError: MGC192.168.201.140@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 1277.733896] LustreError: 66141:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) Skipped 1 previous similar message [ 1277.817101] LustreError: 69491:0:(obd_class.h:479:obd_check_dev()) Device 21 not setup [ 1277.819746] LustreError: 69491:0:(obd_class.h:479:obd_check_dev()) Skipped 5 previous similar messages [ 1277.888912] Lustre: server umount lustre-MDT0001 complete [ 1282.095342] Lustre: DEBUG MARKER: == conf-sanity test 79: format MDT/OST without mgs option (should return errors) ========================================================== 23:14:26 (1763352866) [ 1287.492700] Lustre: DEBUG MARKER: oleg140-server.virtnet: executing load_modules_local [ 1294.584942] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 1306.897703] Lustre: DEBUG MARKER: oleg140-server.virtnet: executing set_hostid [ 1311.459790] Lustre: DEBUG MARKER: oleg140-server.virtnet: executing load_modules_local [ 1315.926200] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 1319.375456] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 1323.317090] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 1327.783577] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 1331.685099] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 1331.718473] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1331.841744] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 1331.866366] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 1331.911530] Lustre: lustre-MDT0000: new disk, initializing [ 1331.958636] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 1331.969799] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 1333.965267] Lustre: DEBUG MARKER: oleg140-server.virtnet: executing set_default_debug -1 all [ 1339.933549] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 1339.967840] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 1340.020101] Lustre: 73734:0:(mgs_llog.c:1346:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 1340.059181] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [ 1340.062169] Lustre: Skipped 1 previous similar message [ 1340.115432] Lustre: lustre-MDT0001: new disk, initializing [ 1340.166186] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 1340.187139] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [ 1340.193589] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 1342.113202] Lustre: DEBUG MARKER: oleg140-server.virtnet: executing set_default_debug -1 all [ 1346.585260] Lustre: DEBUG MARKER: oleg140-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 1348.388561] Lustre: DEBUG MARKER: oleg140-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 1351.776623] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 1351.818463] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 1351.956516] Lustre: lustre-OST0000: new disk, initializing [ 1351.959677] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 1352.008609] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 1354.824444] Lustre: DEBUG MARKER: oleg140-server.virtnet: executing set_default_debug -1 all [ 1358.941873] Lustre: DEBUG MARKER: oleg140-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 1359.885022] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [ 1359.890058] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 1359.946590] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 1361.105914] Lustre: DEBUG MARKER: oleg140-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 [ 1361.210548] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [ 1363.069354] Lustre: DEBUG MARKER: oleg140-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid 50 [ 1363.172200] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [ 1364.964052] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 1364.967681] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 1364.971620] Lustre: Skipped 1 previous similar message [ 1364.976111] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 1364.980264] Lustre: Skipped 2 previous similar messages [ 1366.142141] LustreError: 76671:0:(obd_class.h:479:obd_check_dev()) Device 19 not setup [ 1366.145348] LustreError: 76671:0:(obd_class.h:479:obd_check_dev()) Skipped 7 previous similar messages [ 1366.242836] Lustre: server umount lustre-OST0000 complete [ 1370.083405] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 1370.091471] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 1370.094178] Lustre: Skipped 1 previous similar message [ 1375.201879] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 1375.204407] Lustre: Skipped 1 previous similar message [ 1382.368156] Lustre: lustre-MDT0000 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 1382.423378] LustreError: 76873:0:(obd_class.h:479:obd_check_dev()) Device 22 not setup [ 1382.426061] LustreError: 76873:0:(obd_class.h:479:obd_check_dev()) Skipped 3 previous similar messages [ 1382.470060] Lustre: server umount lustre-MDT0000 complete [ 1384.470702] LustreError: 73727:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1763352969 with bad export cookie 4397838282544781647 [ 1384.473663] LustreError: MGC192.168.201.140@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 1384.477677] LustreError: 73727:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 1384.623089] Lustre: server umount lustre-MDT0001 complete [ 1388.713960] Lustre: DEBUG MARKER: == conf-sanity test 80: mgc import reconnect race ======== 23:16:13 (1763352973) [ 1392.061243] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1392.303314] LustreError: 77746:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 1392.312559] LustreError: 77746:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 3 previous similar messages [ 1392.339276] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 1394.112279] Lustre: DEBUG MARKER: oleg140-server.virtnet: executing set_default_debug -1 all [ 1397.729361] LustreError: 77746:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 1397.950982] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 1399.844029] Lustre: DEBUG MARKER: oleg140-server.virtnet: executing set_default_debug -1 all [ 1402.206337] Lustre: DEBUG MARKER: oleg140-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 1403.787460] Lustre: DEBUG MARKER: oleg140-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 1407.154817] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 1410.206126] Lustre: DEBUG MARKER: oleg140-server.virtnet: executing set_default_debug -1 all [ 1413.149416] Lustre: DEBUG MARKER: oleg140-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 1415.268419] Lustre: 80082:0:(genops.c:1791:obd_export_evict_by_uuid()) MGS: evicting 09079fe9-4bd5-41af-a5ac-d69b261fa7ef at adminstrative request [ 1417.579903] LustreError: MGC192.168.201.140@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 1417.588803] Lustre: *** cfs_fail_loc=906, val=2147483648*** [ 1426.916169] Lustre: MGS: Client 09079fe9-4bd5-41af-a5ac-d69b261fa7ef (at 0@lo) reconnecting [ 1426.920211] Lustre: Evicted from MGS (at 0@lo) after server handle changed from 0x3d08429e02b21cc1 to 0x3d08429e02b21f53 [ 1426.925315] Lustre: MGC192.168.201.140@tcp: Connection restored to 0@lo (at 0@lo) [ 1448.445740] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 1448.479205] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 1448.538802] Lustre: lustre-OST0001: new disk, initializing [ 1448.541706] Lustre: srv-lustre-OST0001: No data found on store. Initialize space. [ 1448.575293] Lustre: lustre-OST0001: Imperative Recovery not enabled, recovery window 60-180 [ 1448.578321] Lustre: Skipped 2 previous similar messages [ 1450.891224] Lustre: DEBUG MARKER: oleg140-server.virtnet: executing set_default_debug -1 all [ 1454.539191] Lustre: DEBUG MARKER: oleg140-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0001-osc-[-0-9a-f]*.ost_server_uuid [ 1455.752075] LustreError: 81207:0:(obd_class.h:479:obd_check_dev()) Device 25 not setup [ 1455.754092] LustreError: 81207:0:(obd_class.h:479:obd_check_dev()) Skipped 13 previous similar messages [ 1455.776582] Lustre: server umount lustre-OST0001 complete [ 1457.656252] LustreError: 79341:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 1457.658924] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 1457.661577] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 1457.661585] Lustre: Skipped 2 previous similar messages [ 1457.675781] Lustre: Skipped 2 previous similar messages [ 1462.757034] LustreError: 79342:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 1462.768667] LustreError: 79342:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 2 previous similar messages [ 1463.517661] Lustre: server umount lustre-OST0000 complete [ 1467.877326] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 1467.883811] Lustre: Skipped 1 previous similar message [ 1471.393405] Lustre: server umount lustre-MDT0000 complete [ 1472.993493] LustreError: 77746:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 1473.003925] LustreError: 77746:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 1 previous similar message [ 1473.061715] LustreError: 80476:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1763353058 with bad export cookie 4397838282544783187 [ 1473.065411] LustreError: MGC192.168.201.140@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 1473.067892] LustreError: 80476:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 1473.172753] LustreError: 81817:0:(obd_class.h:479:obd_check_dev()) Device 23 not setup [ 1473.175894] LustreError: 81817:0:(obd_class.h:479:obd_check_dev()) Skipped 15 previous similar messages [ 1473.259282] Lustre: server umount lustre-MDT0001 complete [ 1476.831408] Lustre: DEBUG MARKER: == conf-sanity test 81: sparse OST indexing ============== 23:17:41 (1763353061) [ 1477.521295] Lustre: DEBUG MARKER: SKIP: conf-sanity test_81 needs >= 3 OSTs [ 1478.362389] Lustre: DEBUG MARKER: == conf-sanity test 82a: specify OSTs for file (succeed) or directory (succeed) ========================================================== 23:17:42 (1763353062) [ 1479.156946] Lustre: DEBUG MARKER: SKIP: conf-sanity test_82a needs >= 3 OSTs [ 1479.984594] Lustre: DEBUG MARKER: == conf-sanity test 82b: specify OSTs for file with --pool and --ost-list options ========================================================== 23:17:44 (1763353064) [ 1480.759285] Lustre: DEBUG MARKER: SKIP: conf-sanity test_82b needs >= 4 OSTs [ 1481.574665] Lustre: DEBUG MARKER: == conf-sanity test 83: ENOSPACE on OST doesn't cause message VFS: Busy inodes after unmount ... ========================================================== 23:17:45 (1763353065) [ 1482.312412] Lustre: DEBUG MARKER: mount the OST /dev/mapper/ost1_flakey as a ldiskfs filesystem [ 1484.917242] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 1485.486654] loop: module loaded [ 1485.507543] loop0: detected capacity change from 0 to 4294967296 [ 1485.542310] LDISKFS-fs (loop0): mounted filesystem with ordered data mode. Opts: (null) [ 1486.540289] Lustre: DEBUG MARKER: run llverfs in partial mode on the OST ldiskfs /mnt/lustre-ost1 [ 1487.986869] Lustre: DEBUG MARKER: oleg140-server.virtnet: executing run_llverfs /mnt/lustre-ost1 -vpl no [ 1489.946383] Lustre: DEBUG MARKER: unmount the OST /dev/mapper/ost1_flakey [ 1493.958094] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 1496.106226] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 1497.957873] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 1499.998361] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 1503.513229] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 1503.558170] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 1503.581092] LustreError: 84386:0:(obd_config.c:783:class_setup()) setup lustre-OST0000-osd failed (-28) [ 1503.586575] LustreError: 84386:0:(obd_mount.c:193:lustre_start_simple()) lustre-OST0000-osd setup error -28 [ 1503.589825] LustreError: 84386:0:(tgt_mount.c:2447:server_fill_super()) Unable to start osd on /dev/mapper/ost1_flakey: -28 [ 1503.593620] LustreError: 84386:0:(super25.c:199:lustre_fill_super()) llite: Unable to mount : rc = -28 [ 1513.127927] Lustre: DEBUG MARKER: oleg140-server.virtnet: executing set_hostid [ 1517.660396] Lustre: DEBUG MARKER: oleg140-server.virtnet: executing load_modules_local [ 1522.374248] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 1525.268326] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 1528.097461] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 1531.204916] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 1534.374792] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 1534.417897] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1534.539184] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 1534.561989] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 1534.605983] Lustre: lustre-MDT0000: new disk, initializing [ 1534.650710] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 1534.660729] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 1536.397483] Lustre: DEBUG MARKER: oleg140-server.virtnet: executing set_default_debug -1 all [ 1541.685433] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 1541.726037] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 1541.769508] Lustre: 87226:0:(mgs_llog.c:1346:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 1541.909407] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 1543.662557] Lustre: DEBUG MARKER: oleg140-server.virtnet: executing set_default_debug -1 all [ 1547.438245] Lustre: DEBUG MARKER: oleg140-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 1548.986307] Lustre: DEBUG MARKER: oleg140-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 1552.050887] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 1552.083338] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 1554.913979] Lustre: DEBUG MARKER: oleg140-server.virtnet: executing set_default_debug -1 all [ 1558.033671] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 1558.107070] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 1559.221901] Lustre: DEBUG MARKER: oleg140-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 1561.193556] Lustre: DEBUG MARKER: oleg140-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 [ 1561.305754] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [ 1563.131639] Lustre: DEBUG MARKER: oleg140-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid 50 [ 1563.230671] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [ 1568.228416] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 1568.230689] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 1568.231966] LustreError: Skipped 1 previous similar message [ 1568.238363] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 1568.242222] Lustre: Skipped 6 previous similar messages [ 1570.159335] LustreError: 90166:0:(obd_class.h:479:obd_check_dev()) Device 19 not setup [ 1570.162389] LustreError: 90166:0:(obd_class.h:479:obd_check_dev()) Skipped 9 previous similar messages [ 1570.213490] Lustre: server umount lustre-OST0000 complete [ 1572.833281] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 1586.144183] Lustre: lustre-MDT0000 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 1586.234788] Lustre: server umount lustre-MDT0000 complete [ 1587.852419] LustreError: 88006:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1763353173 with bad export cookie 4397838282544783740 [ 1587.856172] LustreError: MGC192.168.201.140@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 1587.861080] LustreError: 88006:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 1587.997268] Lustre: server umount lustre-MDT0001 complete [ 1591.757065] Lustre: DEBUG MARKER: == conf-sanity test 84: check recovery_hard_time ========= 23:19:36 (1763353176) [ 1594.899685] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1595.111524] LustreError: 91239:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 1596.975092] Lustre: DEBUG MARKER: oleg140-server.virtnet: executing set_default_debug -1 all [ 1601.263956] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 1603.500179] Lustre: DEBUG MARKER: oleg140-server.virtnet: executing set_default_debug -1 all [ 1605.926476] Lustre: DEBUG MARKER: oleg140-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 1607.571537] Lustre: DEBUG MARKER: oleg140-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 1610.612756] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 1613.108050] Lustre: DEBUG MARKER: oleg140-server.virtnet: executing set_default_debug -1 all [ 1615.412296] Lustre: DEBUG MARKER: oleg140-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 1617.343644] Lustre: DEBUG MARKER: oleg140-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 [ 1620.510490] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 3 sec [ 1622.131637] Lustre: DEBUG MARKER: oleg140-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid 50 [ 1622.220800] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [ 1625.203174] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 1625.235978] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 1625.315469] Lustre: lustre-OST0001: new disk, initializing [ 1625.317512] Lustre: Skipped 2 previous similar messages [ 1625.319863] Lustre: srv-lustre-OST0001: No data found on store. Initialize space. [ 1625.322376] Lustre: Skipped 3 previous similar messages [ 1627.900598] Lustre: DEBUG MARKER: oleg140-server.virtnet: executing set_default_debug -1 all [ 1632.347873] Lustre: DEBUG MARKER: oleg140-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0001-osc-[-0-9a-f]*.ost_server_uuid [ 1633.817296] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x00000002c0000400-0x0000000300000400]:1:ost [ 1633.825478] Lustre: Skipped 2 previous similar messages [ 1633.828137] Lustre: cli-lustre-OST0001-super: Allocated super-sequence [0x00000002c0000400-0x0000000300000400]:1:ost] [ 1633.862468] Lustre: lustre-OST0001-osc-MDT0000: update sequence from 0x100010000 to 0x2c0000401 [ 1634.296721] Lustre: DEBUG MARKER: oleg140-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0001-osc-MDT0000.ost_server_uuid 50 [ 1634.405467] Lustre: DEBUG MARKER: os[cp].lustre-OST0001-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [ 1636.247149] Lustre: DEBUG MARKER: oleg140-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0001-osc-MDT0001.ost_server_uuid 50 [ 1636.362805] Lustre: DEBUG MARKER: os[cp].lustre-OST0001-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [ 1648.032887] Lustre: DEBUG MARKER: mds1 REPLAY BARRIER on lustre-MDT0000 [ 1662.439809] Lustre: Failing over lustre-MDT0000 [ 1662.756580] LustreError: 96165:0:(obd_class.h:479:obd_check_dev()) Device 28 not setup [ 1662.759355] LustreError: 96165:0:(obd_class.h:479:obd_check_dev()) Skipped 17 previous similar messages [ 1662.797487] Lustre: server umount lustre-MDT0000 complete [ 1662.945644] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 1662.949749] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 1662.955566] Lustre: Skipped 3 previous similar messages [ 1662.958143] LustreError: 91951:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 1662.965602] LustreError: 91951:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 1 previous similar message [ 1679.430954] LDISKFS-fs (dm-0): 4 truncates cleaned up [ 1679.433237] LDISKFS-fs (dm-0): recovery complete [ 1679.438394] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1679.505200] LustreError: MGC192.168.201.140@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 1679.623400] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-60 [ 1679.626279] Lustre: Skipped 6 previous similar messages [ 1679.648473] Lustre: lustre-MDT0000: in recovery but waiting for the first client to connect [ 1679.826549] Lustre: lustre-MDT0000: Will be in recovery for at least 1:00, or until 3 clients reconnect [ 1679.826802] Lustre: 91234:0:(ldlm_lib.c:2067:extend_recovery_timer()) lustre-MDT0000: extended recovery timer reached hard limit: 60, extend: 0 [ 1681.329508] Lustre: DEBUG MARKER: oleg140-server.virtnet: executing set_default_debug -1 all [ 1684.964205] Lustre: 91239:0:(ldlm_lib.c:2067:extend_recovery_timer()) lustre-MDT0000: extended recovery timer reached hard limit: 60, extend: 0 [ 1684.969622] Lustre: 91239:0:(ldlm_lib.c:2067:extend_recovery_timer()) Skipped 1 previous similar message [ 1684.975703] Lustre: lustre-MDT0000-lwp-OST0000: Connection restored to 0@lo (at 0@lo) [ 1685.003241] LustreError: 96910:0:(ldlm_lib.c:2679:replay_request_or_update()) cfs_fail_timeout id 709 sleeping for 300ms [ 1685.320130] LustreError: 96910:0:(ldlm_lib.c:2679:replay_request_or_update()) cfs_fail_timeout id 709 awake [ 1685.644141] LustreError: 96910:0:(ldlm_lib.c:2679:replay_request_or_update()) cfs_fail_timeout id 709 sleeping for 300ms [ 1685.662143] LustreError: 96910:0:(ldlm_lib.c:2679:replay_request_or_update()) Skipped 1 previous similar message [ 1685.976179] LustreError: 96910:0:(ldlm_lib.c:2679:replay_request_or_update()) cfs_fail_timeout id 709 awake [ 1685.980073] LustreError: 96910:0:(ldlm_lib.c:2679:replay_request_or_update()) Skipped 1 previous similar message [ 1685.984787] Lustre: 96910:0:(ldlm_lib.c:2067:extend_recovery_timer()) lustre-MDT0000: extended recovery timer reached hard limit: 60, extend: 1 [ 1685.990486] Lustre: 96910:0:(ldlm_lib.c:2067:extend_recovery_timer()) Skipped 7 previous similar messages [ 1686.939400] LustreError: 96910:0:(ldlm_lib.c:2679:replay_request_or_update()) cfs_fail_timeout id 709 sleeping for 300ms [ 1686.943748] LustreError: 96910:0:(ldlm_lib.c:2679:replay_request_or_update()) Skipped 3 previous similar messages [ 1687.256244] LustreError: 96910:0:(ldlm_lib.c:2679:replay_request_or_update()) cfs_fail_timeout id 709 awake [ 1687.259977] LustreError: 96910:0:(ldlm_lib.c:2679:replay_request_or_update()) Skipped 3 previous similar messages [ 1688.202894] Lustre: 96910:0:(ldlm_lib.c:2067:extend_recovery_timer()) lustre-MDT0000: extended recovery timer reached hard limit: 60, extend: 1 [ 1688.208264] Lustre: 96910:0:(ldlm_lib.c:2067:extend_recovery_timer()) Skipped 6 previous similar messages [ 1689.145239] LustreError: 96910:0:(ldlm_lib.c:2679:replay_request_or_update()) cfs_fail_timeout id 709 sleeping for 300ms [ 1689.149323] LustreError: 96910:0:(ldlm_lib.c:2679:replay_request_or_update()) Skipped 6 previous similar messages [ 1689.464136] LustreError: 96910:0:(ldlm_lib.c:2679:replay_request_or_update()) cfs_fail_timeout id 709 awake [ 1689.468422] LustreError: 96910:0:(ldlm_lib.c:2679:replay_request_or_update()) Skipped 6 previous similar messages [ 1692.297067] Lustre: 96910:0:(ldlm_lib.c:2067:extend_recovery_timer()) lustre-MDT0000: extended recovery timer reached hard limit: 60, extend: 1 [ 1692.307149] Lustre: 96910:0:(ldlm_lib.c:2067:extend_recovery_timer()) Skipped 12 previous similar messages [ 1693.258721] LustreError: 96910:0:(ldlm_lib.c:2679:replay_request_or_update()) cfs_fail_timeout id 709 sleeping for 300ms [ 1693.263174] LustreError: 96910:0:(ldlm_lib.c:2679:replay_request_or_update()) Skipped 12 previous similar messages [ 1693.576133] LustreError: 96910:0:(ldlm_lib.c:2679:replay_request_or_update()) cfs_fail_timeout id 709 awake [ 1693.579445] LustreError: 96910:0:(ldlm_lib.c:2679:replay_request_or_update()) Skipped 12 previous similar messages [ 1700.457207] Lustre: 96910:0:(ldlm_lib.c:2067:extend_recovery_timer()) lustre-MDT0000: extended recovery timer reached hard limit: 60, extend: 1 [ 1700.462591] Lustre: 96910:0:(ldlm_lib.c:2067:extend_recovery_timer()) Skipped 25 previous similar messages [ 1701.402700] LustreError: 96910:0:(ldlm_lib.c:2679:replay_request_or_update()) cfs_fail_timeout id 709 sleeping for 300ms [ 1701.406158] LustreError: 96910:0:(ldlm_lib.c:2679:replay_request_or_update()) Skipped 25 previous similar messages [ 1701.728192] LustreError: 96910:0:(ldlm_lib.c:2679:replay_request_or_update()) cfs_fail_timeout id 709 awake [ 1701.732750] LustreError: 96910:0:(ldlm_lib.c:2679:replay_request_or_update()) Skipped 25 previous similar messages [ 1716.753223] Lustre: 96910:0:(ldlm_lib.c:2067:extend_recovery_timer()) lustre-MDT0000: extended recovery timer reached hard limit: 60, extend: 1 [ 1716.760303] Lustre: 96910:0:(ldlm_lib.c:2067:extend_recovery_timer()) Skipped 51 previous similar messages [ 1717.706760] LustreError: 96910:0:(ldlm_lib.c:2679:replay_request_or_update()) cfs_fail_timeout id 709 sleeping for 300ms [ 1717.711107] LustreError: 96910:0:(ldlm_lib.c:2679:replay_request_or_update()) Skipped 51 previous similar messages [ 1718.024187] LustreError: 96910:0:(ldlm_lib.c:2679:replay_request_or_update()) cfs_fail_timeout id 709 awake [ 1718.027856] LustreError: 96910:0:(ldlm_lib.c:2679:replay_request_or_update()) Skipped 51 previous similar messages [ 1739.264223] Lustre: 96910:0:(ldlm_lib.c:2375:target_recovery_overseer()) lustre-MDT0000 recovery is aborted by hard timeout [ 1739.268314] Lustre: 96910:0:(ldlm_lib.c:2385:target_recovery_overseer()) recovery is aborted, evict exports in recovery [ 1739.272289] Lustre: 96910:0:(genops.c:1618:class_disconnect_stale_exports()) lustre-MDT0000: disconnect stale client d1addc7c-e82a-4b23-b018-8afcd49be3eb@192.168.201.40@tcp [ 1739.278557] Lustre: lustre-MDT0000: disconnecting 1 stale clients [ 1739.281640] Lustre: 96910:0:(ldlm_lib.c:1897:abort_req_replay_queue()) @@@ aborted: req@00000000afbe207b x1849009405756288/t0(8589934770) o35->d1addc7c-e82a-4b23-b018-8afcd49be3eb@192.168.201.40@tcp:250/0 lens 392/0 e 0 to 0 dl 1763353335 ref 1 fl Complete:/604/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:0 [ 1739.294823] Lustre: lustre-MDT0000-osd: cancel update llog [0x200000400:0x1:0x0] [ 1739.299815] Lustre: lustre-MDT0000: Denying connection for new client d1addc7c-e82a-4b23-b018-8afcd49be3eb (at 192.168.201.40@tcp), waiting for 3 known clients (2 recovered, 0 in progress, and 1 evicted) already passed deadline 0:00 [ 1739.305692] Lustre: lustre-MDT0001-osp-MDT0000: cancel update llog [0x240000401:0x1:0x0] [ 1739.335860] Lustre: lustre-MDT0000-osp-MDT0001: Connection restored to 0@lo (at 0@lo) [ 1739.340948] Lustre: lustre-MDT0000: Recovery over after 1:00, of 3 clients 2 recovered and 1 was evicted. [ 1739.341657] Lustre: Skipped 2 previous similar messages [ 1739.364533] Lustre: lustre-OST0001: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x2c0000401:46 to 0x2c0000401:545) [ 1739.364593] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:77 to 0x280000401:577) [ 1749.988231] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 1749.988467] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 1749.991790] LustreError: Skipped 1 previous similar message [ 1749.992931] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 1749.998173] Lustre: Skipped 4 previous similar messages [ 1750.004839] Lustre: Skipped 8 previous similar messages [ 1753.478946] Lustre: server umount lustre-OST0000 complete [ 1756.642678] LustreError: 92836:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-OST0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 1756.650460] LustreError: 92836:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 21 previous similar messages [ 1761.251857] Lustre: server umount lustre-OST0001 complete [ 1769.137501] Lustre: server umount lustre-MDT0000 complete [ 1770.974854] LustreError: 91220:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1763353356 with bad export cookie 4397838282544834938 [ 1770.976929] LustreError: MGC192.168.201.140@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 1770.980390] LustreError: 91220:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 1771.136065] Lustre: server umount lustre-MDT0001 complete [ 1775.212776] Lustre: DEBUG MARKER: == conf-sanity test 85: osd_ost init: fail ea_fid_set ==== 23:22:39 (1763353359) [ 1778.917759] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 1778.921686] Lustre: *** cfs_fail_loc=197, val=0*** [ 1794.400654] LustreError: Failed to get MGS log lustre-sptlrpc and no local copy. [ 1799.520891] LustreError: 98864:0:(mgc_request.c:233:do_config_log_add()) MGC192.168.201.140@tcp: failed processing log, type 4: rc = -110 [ 1804.659464] Lustre: *** cfs_fail_loc=197, val=0*** [ 1804.661792] Lustre: Skipped 65 previous similar messages [ 1827.538108] Lustre: DEBUG MARKER: oleg140-server.virtnet: executing set_default_debug -1 all [ 1829.955261] Lustre: DEBUG MARKER: oleg140-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 1830.894973] LustreError: 99528:0:(obd_class.h:479:obd_check_dev()) Device 1 not setup [ 1830.897559] LustreError: 99528:0:(obd_class.h:479:obd_check_dev()) Skipped 33 previous similar messages [ 1830.941263] Lustre: server umount lustre-OST0000 complete [ 1834.822970] Lustre: DEBUG MARKER: == conf-sanity test 86: Replacing mkfs.lustre -G option == 23:23:39 (1763353419) [ 1850.121390] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1850.357503] LustreError: 100372:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 1852.250921] Lustre: DEBUG MARKER: oleg140-server.virtnet: executing set_default_debug -1 all [ 1856.132734] Lustre: DEBUG MARKER: oleg140-client.virtnet: executing wait_import_state_mount (FULL|IDLE) mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 1858.378046] LustreError: 100402:0:(osp_object.c:617:osp_attr_get()) lustre-MDT0001-osp-MDT0000: osp_attr_get update error [0x200000009:0x1:0x0]: rc = -5 [ 1858.382734] LustreError: 100402:0:(lod_sub_object.c:917:lod_sub_prep_llog()) lustre-MDT0000-mdtlov: can't get id from catalogs: rc = -5 [ 1858.386692] LustreError: 100402:0:(lod_dev.c:510:lod_sub_recovery_thread()) lustre-MDT0001-osp-MDT0000: get update log duration 8, retries 0, failed: rc = -5 [ 1858.511144] Lustre: server umount lustre-MDT0000 complete [ 1864.859936] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 1872.236583] Lustre: DEBUG MARKER: oleg140-server.virtnet: executing set_hostid [ 1876.305224] Lustre: DEBUG MARKER: oleg140-server.virtnet: executing load_modules_local [ 1880.813845] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 1883.652493] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 1886.217259] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 1888.924469] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 1891.770778] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 1891.801549] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1891.916878] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 1891.935180] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 1891.977879] Lustre: lustre-MDT0000: new disk, initializing [ 1892.031829] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 1893.822098] Lustre: DEBUG MARKER: oleg140-server.virtnet: executing set_default_debug -1 all [ 1899.912331] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 1899.955240] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 1900.015093] Lustre: 104434:0:(mgs_llog.c:1346:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 1900.212276] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 1902.404175] Lustre: DEBUG MARKER: oleg140-server.virtnet: executing set_default_debug -1 all [ 1906.723132] Lustre: DEBUG MARKER: oleg140-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 1908.577686] Lustre: DEBUG MARKER: oleg140-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 1911.790502] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 1911.823140] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 1914.761249] Lustre: DEBUG MARKER: oleg140-server.virtnet: executing set_default_debug -1 all [ 1918.705197] Lustre: DEBUG MARKER: oleg140-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 1920.450311] Lustre: DEBUG MARKER: oleg140-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 [ 1920.562300] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 1921.578158] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 1 sec [ 1923.303351] Lustre: DEBUG MARKER: oleg140-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid 50 [ 1923.402723] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [ 1925.601962] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 1925.603336] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 1925.617847] LustreError: Skipped 2 previous similar messages [ 1925.623755] Lustre: Skipped 4 previous similar messages [ 1925.628350] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 1925.630923] Lustre: Skipped 7 previous similar messages [ 1926.321104] Lustre: server umount lustre-OST0000 complete [ 1934.192315] Lustre: server umount lustre-MDT0000 complete [ 1935.970149] LustreError: 104427:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1763353521 with bad export cookie 4397838282544845501 [ 1935.972923] LustreError: MGC192.168.201.140@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 1935.976873] LustreError: 104427:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) Skipped 2 previous similar messages [ 1936.152062] Lustre: server umount lustre-MDT0001 complete [ 1940.034811] Lustre: DEBUG MARKER: == conf-sanity test 87: check if MDT inode can hold EAs with N stripes properly ========================================================== 23:25:24 (1763353524) [ 1944.006566] Lustre: DEBUG MARKER: oleg140-server.virtnet: executing unload_modules_local [ 1945.412300] Key type lgssc unregistered [ 1945.600523] LNet: 108510:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 1945.607208] LNetError: 108510:0:(acceptor.c:264:lnet_acceptor_remove_socket()) Interface ens2 not found [ 1945.618033] LNet: Removed LNI 192.168.201.140@tcp [ 1946.083336] Key type .llcrypt unregistered [ 1946.085092] Key type ._llcrypt unregistered [ 1953.581703] Key type ._llcrypt registered [ 1953.582945] Key type .llcrypt registered [ 1953.628662] Lustre: DEBUG MARKER: oleg140-server.virtnet: executing set_hostid [ 1960.875986] Lustre: DEBUG MARKER: oleg140-server.virtnet: executing load_modules_local [ 1961.339619] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 1961.355412] alg: No test for adler32 (adler32-zlib) [ 1962.271237] Lustre: Lustre: Build Version: 2.16.61_41_g6a6a76f [ 1962.375799] LNet: Added LNI 192.168.201.140@tcp [8/256/0/180] [ 1963.976246] Key type lgssc registered [ 1964.474736] Lustre: Echo OBD driver; http://www.lustre.org/ [ 1968.338292] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 1971.203210] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 1973.631896] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 1976.409250] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 1978.838122] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 1981.958267] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 1981.983980] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 1981.992352] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1983.108303] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 1983.124248] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 1983.163907] Lustre: lustre-MDT0000: new disk, initializing [ 1983.197135] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 1983.206950] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 1984.852264] Lustre: DEBUG MARKER: oleg140-server.virtnet: executing set_default_debug -1 all [ 1989.733506] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 1989.763277] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 1989.878786] Lustre: lustre-OST0000: new disk, initializing [ 1989.881798] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 1989.884301] Lustre: Skipped 1 previous similar message [ 1989.917488] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 1992.124413] Lustre: DEBUG MARKER: oleg140-server.virtnet: executing set_default_debug -1 all [ 1996.820253] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:0:ost [ 1996.825491] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:0:ost] [ 1996.842946] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x240000400 [ 1997.228827] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 1997.257798] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 1997.320557] Lustre: lustre-OST0001: new disk, initializing [ 1997.325227] Lustre: srv-lustre-OST0001: No data found on store. Initialize space. [ 1997.362540] Lustre: lustre-OST0001: Imperative Recovery not enabled, recovery window 60-180 [ 1999.718171] Lustre: DEBUG MARKER: oleg140-server.virtnet: executing set_default_debug -1 all [ 2007.522363] Lustre: lustre-MDT0000-lwp-OST0000: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2007.530872] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 2007.534695] Lustre: Skipped 1 previous similar message [ 2010.451227] LustreError: 114059:0:(obd_class.h:479:obd_check_dev()) Device 13 not setup [ 2010.529766] Lustre: server umount lustre-MDT0000 complete [ 2013.551644] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: (null) [ 2015.859462] LustreError: 112620:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1763353601 with bad export cookie 8870892938671508166 [ 2015.861420] LustreError: MGC192.168.201.140@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 2015.864509] LustreError: 112620:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) Skipped 2 previous similar messages [ 2015.885140] LustreError: 114656:0:(obd_class.h:479:obd_check_dev()) Device 12 not setup [ 2015.888370] LustreError: 114656:0:(obd_class.h:479:obd_check_dev()) Skipped 3 previous similar messages [ 2015.910205] Lustre: server umount lustre-OST0000 complete [ 2017.668113] LustreError: 114856:0:(obd_class.h:479:obd_check_dev()) Device 2 not setup [ 2017.671658] LustreError: 114856:0:(obd_class.h:479:obd_check_dev()) Skipped 1 previous similar message [ 2017.733583] Lustre: server umount lustre-OST0001 complete [ 2021.654328] Lustre: DEBUG MARKER: == conf-sanity test 88: check the default mount options can be overridden ========================================================== 23:26:46 (1763353606) [ 2025.500818] Lustre: DEBUG MARKER: oleg140-server.virtnet: executing load_modules_local [ 2029.445162] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 2029.902528] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 2032.430283] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 2032.911279] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 2035.974484] Lustre: DEBUG MARKER: == conf-sanity test 89: check tunefs --param and --erase-param{s} options ========================================================== 23:27:00 (1763353620) [ 2041.602070] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 2042.431552] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 2043.205312] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 2044.051667] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 2044.461840] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 2045.417542] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 2053.019779] Lustre: DEBUG MARKER: oleg140-server.virtnet: executing set_hostid [ 2057.547468] Lustre: DEBUG MARKER: oleg140-server.virtnet: executing load_modules_local [ 2061.850626] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 2064.766506] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 2067.457168] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 2070.214602] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 2072.954938] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 2072.989192] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 2073.098366] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 2073.117980] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 2073.160366] Lustre: lustre-MDT0000: new disk, initializing [ 2073.206820] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 2073.216561] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 2074.852911] Lustre: DEBUG MARKER: oleg140-server.virtnet: executing set_default_debug -1 all [ 2079.719659] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 2079.744634] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 2079.786709] Lustre: 120170:0:(mgs_llog.c:1346:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 2079.815926] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [ 2079.818526] Lustre: Skipped 1 previous similar message [ 2079.856920] Lustre: lustre-MDT0001: new disk, initializing [ 2079.899513] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 2079.921219] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [ 2079.927438] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 2081.595269] Lustre: DEBUG MARKER: oleg140-server.virtnet: executing set_default_debug -1 all [ 2085.308952] Lustre: DEBUG MARKER: oleg140-client.virtnet: executing wait_import_state_mount FULL failover.node=192.0.2.254@tcp0,mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 2086.867493] Lustre: DEBUG MARKER: oleg140-client.virtnet: executing wait_import_state_mount FULL failover.node=192.0.2.254@tcp0,mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 2090.054147] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 2090.089708] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 2090.216093] Lustre: lustre-OST0000: new disk, initializing [ 2090.218901] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 2090.255981] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 2092.816630] Lustre: DEBUG MARKER: oleg140-server.virtnet: executing set_default_debug -1 all [ 2096.399907] Lustre: DEBUG MARKER: oleg140-client.virtnet: executing wait_import_state_mount (FULL|IDLE) failover.node=192.0.2.254@tcp0,osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 2096.639795] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [ 2096.643635] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 2096.689789] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 2098.028175] Lustre: DEBUG MARKER: oleg140-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 [ 2098.115619] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [ 2099.670319] Lustre: DEBUG MARKER: oleg140-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid 50 [ 2099.760623] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [ 2101.730742] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 2101.732617] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2101.738569] LustreError: Skipped 1 previous similar message [ 2101.747514] Lustre: Skipped 1 previous similar message [ 2101.750747] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 2101.754782] Lustre: Skipped 1 previous similar message [ 2102.639282] LustreError: 123109:0:(obd_class.h:479:obd_check_dev()) Device 19 not setup [ 2102.641376] LustreError: 123109:0:(obd_class.h:479:obd_check_dev()) Skipped 3 previous similar messages [ 2102.691761] Lustre: server umount lustre-OST0000 complete [ 2105.825105] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 2105.827533] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2105.833112] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 2110.474627] LustreError: 123311:0:(obd_class.h:479:obd_check_dev()) Device 22 not setup [ 2110.476691] LustreError: 123311:0:(obd_class.h:479:obd_check_dev()) Skipped 3 previous similar messages [ 2110.516255] Lustre: server umount lustre-MDT0000 complete [ 2112.268924] LustreError: 120951:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1763353697 with bad export cookie 8870892938671509314 [ 2112.270989] LustreError: MGC192.168.201.140@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 2112.273906] LustreError: 120951:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) Skipped 2 previous similar messages [ 2112.404392] Lustre: server umount lustre-MDT0001 complete [ 2117.056339] Lustre: DEBUG MARKER: == conf-sanity test 90a: check max_mod_rpcs_in_flight is enforced ========================================================== 23:28:21 (1763353701) [ 2120.299351] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 2120.519435] LustreError: 124182:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 2120.527147] LustreError: 124182:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 3 previous similar messages [ 2120.556195] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 2122.326260] Lustre: DEBUG MARKER: oleg140-server.virtnet: executing set_default_debug -1 all [ 2125.794360] LustreError: 124182:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 2126.294680] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 2128.202927] Lustre: DEBUG MARKER: oleg140-server.virtnet: executing set_default_debug -1 all [ 2130.507707] Lustre: DEBUG MARKER: oleg140-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 2132.230872] Lustre: DEBUG MARKER: oleg140-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 2135.396453] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 2138.025818] Lustre: DEBUG MARKER: oleg140-server.virtnet: executing set_default_debug -1 all [ 2140.700650] Lustre: DEBUG MARKER: oleg140-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 2148.495034] Lustre: *** cfs_fail_loc=159, val=0*** [ 2164.697699] Lustre: lustre-MDT0000: Client d55fc7c6-4272-4ce3-a889-5395c493b638 (at 192.168.201.40@tcp) reconnecting [ 2165.425336] Lustre: *** cfs_fail_loc=159, val=0*** [ 2165.433122] Lustre: Skipped 6 previous similar messages [ 2181.083932] Lustre: lustre-MDT0000: Client d55fc7c6-4272-4ce3-a889-5395c493b638 (at 192.168.201.40@tcp) reconnecting [ 2184.675220] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2184.681046] Lustre: Skipped 1 previous similar message [ 2184.684939] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 2184.688017] Lustre: Skipped 2 previous similar messages [ 2189.296173] LustreError: 126669:0:(obd_class.h:479:obd_check_dev()) Device 21 not setup [ 2189.299476] LustreError: 126669:0:(obd_class.h:479:obd_check_dev()) Skipped 13 previous similar messages [ 2189.341892] Lustre: server umount lustre-OST0000 complete [ 2193.377036] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 2193.380160] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2193.386187] Lustre: Skipped 1 previous similar message [ 2193.391985] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 2197.099200] Lustre: server umount lustre-MDT0000 complete [ 2198.757082] LustreError: 124163:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1763353783 with bad export cookie 8870892938671510182 [ 2198.760854] LustreError: MGC192.168.201.140@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 2198.763337] LustreError: 124163:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 2198.895664] Lustre: server umount lustre-MDT0001 complete [ 2203.891904] Lustre: DEBUG MARKER: oleg140-server.virtnet: executing unload_modules_local [ 2205.096515] Key type lgssc unregistered [ 2205.240340] LNet: 127604:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 2205.243938] LNetError: 127604:0:(acceptor.c:264:lnet_acceptor_remove_socket()) Interface ens2 not found [ 2205.255691] LNet: Removed LNI 192.168.201.140@tcp [ 2205.622125] Key type .llcrypt unregistered [ 2205.623509] Key type ._llcrypt unregistered [ 2217.326415] Key type ._llcrypt registered [ 2217.328104] Key type .llcrypt registered [ 2217.374808] Lustre: DEBUG MARKER: oleg140-server.virtnet: executing load_modules_local [ 2217.755670] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 2217.814471] alg: No test for adler32 (adler32-zlib) [ 2218.695891] Lustre: Lustre: Build Version: 2.16.61_41_g6a6a76f [ 2218.802386] LNet: Added LNI 192.168.201.140@tcp [8/256/0/180] [ 2220.392210] Key type lgssc registered [ 2220.854560] Lustre: Echo OBD driver; http://www.lustre.org/ [ 2223.996413] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 2224.004170] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 2225.202100] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 2226.791945] Lustre: DEBUG MARKER: oleg140-server.virtnet: executing set_default_debug -1 all [ 2230.461243] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 2230.627942] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 2232.255103] Lustre: DEBUG MARKER: oleg140-server.virtnet: executing set_default_debug -1 all [ 2234.608482] Lustre: DEBUG MARKER: oleg140-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 2236.090095] Lustre: DEBUG MARKER: oleg140-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 2238.883054] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 2239.046706] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 2241.463546] Lustre: DEBUG MARKER: oleg140-server.virtnet: executing set_default_debug -1 all [ 2243.673701] Lustre: DEBUG MARKER: oleg140-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 2250.346778] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:42 to 0x280000401:65) [ 2259.414932] Lustre: *** cfs_fail_loc=159, val=0*** [ 2275.794377] Lustre: lustre-MDT0000: Client 58c0a3af-ca41-4252-bbe1-73fffad8db60 (at 192.168.201.40@tcp) reconnecting [ 2276.782184] Lustre: *** cfs_fail_loc=159, val=0*** [ 2292.181899] Lustre: lustre-MDT0001: Client 58c0a3af-ca41-4252-bbe1-73fffad8db60 (at 192.168.201.40@tcp) reconnecting [ 2292.853808] Lustre: *** cfs_fail_loc=159, val=0*** [ 2292.856872] Lustre: Skipped 4 previous similar messages [ 2308.050652] Lustre: lustre-MDT0001: Client 58c0a3af-ca41-4252-bbe1-73fffad8db60 (at 192.168.201.40@tcp) reconnecting [ 2309.651713] Lustre: 131702:0:(mdt_handler.c:64:mdt_max_mod_rpcs_per_client_set()) max_mod_rpcs_per_client is deprecated, set mdt.*.max_mod_rpcs_in_flight parameter instead [ 2310.942317] Lustre: *** cfs_fail_loc=159, val=0*** [ 2310.946307] Lustre: Skipped 4 previous similar messages [ 2326.491882] Lustre: lustre-MDT0000: Client 4d82181b-06e2-408d-88fe-6f7f01a1c245 (at 192.168.201.40@tcp) reconnecting [ 2327.248049] Lustre: *** cfs_fail_loc=159, val=0*** [ 2327.250095] Lustre: Skipped 13 previous similar messages [ 2343.889987] Lustre: lustre-MDT0000: Client 4d82181b-06e2-408d-88fe-6f7f01a1c245 (at 192.168.201.40@tcp) reconnecting [ 2344.497892] Lustre: 131992:0:(mdt_handler.c:64:mdt_max_mod_rpcs_per_client_set()) max_mod_rpcs_per_client is deprecated, set mdt.*.max_mod_rpcs_in_flight parameter instead [ 2348.005523] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 2348.007460] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2348.009389] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 2348.014028] Lustre: Skipped 1 previous similar message [ 2351.010783] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 2351.013603] Lustre: Skipped 2 previous similar messages [ 2352.307177] LustreError: 132089:0:(obd_class.h:479:obd_check_dev()) Device 21 not setup [ 2352.359432] Lustre: server umount lustre-OST0000 complete [ 2356.194132] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2356.194712] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 2356.199938] Lustre: Skipped 1 previous similar message [ 2360.321879] LustreError: 132290:0:(obd_class.h:479:obd_check_dev()) Device 9 not setup [ 2360.324077] LustreError: 132290:0:(obd_class.h:479:obd_check_dev()) Skipped 3 previous similar messages [ 2360.354706] Lustre: server umount lustre-MDT0000 complete [ 2361.315174] LustreError: 128978:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 2361.321885] LustreError: 128978:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 3 previous similar messages [ 2361.805187] LustreError: 128959:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1763353946 with bad export cookie 5611473671580475711 [ 2361.807482] LustreError: MGC192.168.201.140@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 2361.810581] LustreError: 128959:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) Skipped 3 previous similar messages [ 2361.882153] LustreError: 132491:0:(obd_class.h:479:obd_check_dev()) Device 16 not setup [ 2361.884963] LustreError: 132491:0:(obd_class.h:479:obd_check_dev()) Skipped 5 previous similar messages [ 2361.946122] Lustre: server umount lustre-MDT0001 complete [ 2365.959842] Lustre: DEBUG MARKER: oleg140-server.virtnet: executing unload_modules_local [ 2367.072608] Key type lgssc unregistered [ 2367.216418] LNet: 133022:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 2367.219377] LNetError: 133022:0:(acceptor.c:264:lnet_acceptor_remove_socket()) Interface ens2 not found [ 2367.228401] LNet: Removed LNI 192.168.201.140@tcp [ 2367.559605] Key type .llcrypt unregistered [ 2367.560685] Key type ._llcrypt unregistered [ 2377.661305] Key type ._llcrypt registered [ 2377.662464] Key type .llcrypt registered [ 2377.702961] Lustre: DEBUG MARKER: oleg140-server.virtnet: executing load_modules_local [ 2378.021340] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 2378.029542] alg: No test for adler32 (adler32-zlib) [ 2378.898548] Lustre: Lustre: Build Version: 2.16.61_41_g6a6a76f [ 2378.986480] LNet: Added LNI 192.168.201.140@tcp [8/256/0/180] [ 2380.568153] Key type lgssc registered [ 2380.955432] Lustre: Echo OBD driver; http://www.lustre.org/ [ 2383.968285] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 2383.976544] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 2385.185389] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 2386.762535] Lustre: DEBUG MARKER: oleg140-server.virtnet: executing set_default_debug -1 all [ 2390.265494] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 2390.430992] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 2391.919485] Lustre: DEBUG MARKER: oleg140-server.virtnet: executing set_default_debug -1 all [ 2393.974152] Lustre: DEBUG MARKER: oleg140-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 2395.399602] Lustre: DEBUG MARKER: oleg140-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 2398.236954] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 2398.370659] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 2400.426265] Lustre: lustre-OST0000: new connection from lustre-MDT0001-mdtlov (cleaning up unused objects from 0x280000400:72 to 0x280000400:97) [ 2400.580121] Lustre: DEBUG MARKER: oleg140-server.virtnet: executing set_default_debug -1 all [ 2402.616747] Lustre: DEBUG MARKER: oleg140-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 2410.667016] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:85 to 0x280000401:129) [ 2415.681841] Lustre: 136665:0:(mdt_handler.c:64:mdt_max_mod_rpcs_per_client_set()) max_mod_rpcs_per_client is deprecated, set mdt.*.max_mod_rpcs_in_flight parameter instead [ 2416.722854] Lustre: 136714:0:(mdt_handler.c:64:mdt_max_mod_rpcs_per_client_set()) max_mod_rpcs_per_client is deprecated, set mdt.*.max_mod_rpcs_in_flight parameter instead [ 2417.340929] Lustre: Modifying parameter lustre.mdc.lustre-MDT0000-mdc-*.max_rpcs_in_flight=8 in log params [ 2418.816243] Lustre: 136863:0:(mgs_llog.c:1346:mgs_modify_param()) MGS: modify lustre/mdc.lustre-MDT0000-mdc-*.max_rpcs_in_flight=8 (mode = 0) failed: rc = -17 [ 2419.214513] Lustre: Modifying parameter lustre.mdc.lustre-MDT0000-mdc-*.max_mod_rpcs_in_flight=7 in log params [ 2419.218694] Lustre: Skipped 1 previous similar message [ 2419.601709] Lustre: 136961:0:(mdt_handler.c:64:mdt_max_mod_rpcs_per_client_set()) max_mod_rpcs_per_client is deprecated, set mdt.*.max_mod_rpcs_in_flight parameter instead [ 2421.218881] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 2421.222628] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2421.224673] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 2421.228722] Lustre: Skipped 1 previous similar message [ 2422.904109] LustreError: 137059:0:(obd_class.h:479:obd_check_dev()) Device 21 not setup [ 2422.958154] Lustre: server umount lustre-OST0000 complete [ 2426.338140] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 2426.338483] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2426.339562] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 2426.339568] Lustre: Skipped 1 previous similar message [ 2426.341375] LustreError: Skipped 1 previous similar message [ 2426.353042] Lustre: Skipped 1 previous similar message [ 2426.753108] LustreError: 137260:0:(obd_class.h:479:obd_check_dev()) Device 9 not setup [ 2426.755682] LustreError: 137260:0:(obd_class.h:479:obd_check_dev()) Skipped 3 previous similar messages [ 2426.786532] Lustre: server umount lustre-MDT0000 complete [ 2428.434538] LustreError: 136677:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1763354013 with bad export cookie 11085739359096709954 [ 2428.435388] LustreError: MGC192.168.201.140@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 2428.438735] LustreError: 136677:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) Skipped 2 previous similar messages [ 2428.442269] LustreError: 134387:0:(mgc_request.c:614:do_requeue()) failed processing log: -5 [ 2428.521452] LustreError: 137460:0:(obd_class.h:479:obd_check_dev()) Device 16 not setup [ 2428.524057] LustreError: 137460:0:(obd_class.h:479:obd_check_dev()) Skipped 5 previous similar messages [ 2428.605897] Lustre: server umount lustre-MDT0001 complete [ 2432.919334] Lustre: DEBUG MARKER: oleg140-server.virtnet: executing unload_modules_local [ 2434.270187] Key type lgssc unregistered [ 2434.420766] LNet: 137991:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 2434.425799] LNetError: 137991:0:(acceptor.c:264:lnet_acceptor_remove_socket()) Interface ens2 not found [ 2434.440492] LNet: Removed LNI 192.168.201.140@tcp [ 2434.808490] Key type .llcrypt unregistered [ 2434.810152] Key type ._llcrypt unregistered [ 2445.291708] Key type ._llcrypt registered [ 2445.293410] Key type .llcrypt registered [ 2445.344416] Lustre: DEBUG MARKER: oleg140-server.virtnet: executing load_modules_local [ 2445.788601] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 2445.799371] alg: No test for adler32 (adler32-zlib) [ 2446.680252] Lustre: Lustre: Build Version: 2.16.61_41_g6a6a76f [ 2446.784691] LNet: Added LNI 192.168.201.140@tcp [8/256/0/180] [ 2448.376366] Key type lgssc registered [ 2448.835577] Lustre: Echo OBD driver; http://www.lustre.org/ [ 2452.097138] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 2452.105122] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 2453.311657] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 2454.864783] Lustre: DEBUG MARKER: oleg140-server.virtnet: executing set_default_debug -1 all [ 2458.312655] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 2458.490913] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 2460.114787] Lustre: DEBUG MARKER: oleg140-server.virtnet: executing set_default_debug -1 all [ 2462.305868] Lustre: DEBUG MARKER: oleg140-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 2463.732726] Lustre: DEBUG MARKER: oleg140-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 2466.552074] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 2466.720776] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 2468.776727] Lustre: lustre-OST0000: new connection from lustre-MDT0001-mdtlov (cleaning up unused objects from 0x280000400:72 to 0x280000400:129) [ 2469.074162] Lustre: DEBUG MARKER: oleg140-server.virtnet: executing set_default_debug -1 all [ 2471.295496] Lustre: DEBUG MARKER: oleg140-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 2477.993414] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:85 to 0x280000401:161) [ 2484.519376] Lustre: *** cfs_fail_loc=159, val=0*** [ 2500.049720] Lustre: lustre-MDT0000: Client 44182cec-d08e-4561-919c-f23e90020255 (at 192.168.201.40@tcp) reconnecting [ 2519.011674] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2519.012524] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 2519.016660] Lustre: Skipped 1 previous similar message [ 2519.020894] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 2522.165755] LustreError: 141771:0:(obd_class.h:479:obd_check_dev()) Device 21 not setup [ 2522.220345] Lustre: server umount lustre-OST0000 complete [ 2524.129537] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2524.136581] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 2524.138842] Lustre: Skipped 2 previous similar messages [ 2529.249721] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 2529.254522] Lustre: Skipped 1 previous similar message [ 2530.246844] LustreError: 141974:0:(obd_class.h:479:obd_check_dev()) Device 9 not setup [ 2530.250109] LustreError: 141974:0:(obd_class.h:479:obd_check_dev()) Skipped 3 previous similar messages [ 2530.299315] Lustre: server umount lustre-MDT0000 complete [ 2531.968332] LustreError: 140991:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1763354117 with bad export cookie 10018638731692140005 [ 2531.971410] LustreError: MGC192.168.201.140@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 2531.973750] LustreError: 140991:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 2532.048534] LustreError: 142175:0:(obd_class.h:479:obd_check_dev()) Device 16 not setup [ 2532.051556] LustreError: 142175:0:(obd_class.h:479:obd_check_dev()) Skipped 5 previous similar messages [ 2532.109396] Lustre: server umount lustre-MDT0001 complete [ 2536.355453] Lustre: DEBUG MARKER: oleg140-server.virtnet: executing unload_modules_local [ 2537.574792] Key type lgssc unregistered [ 2537.727568] LNet: 142708:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 2537.732180] LNetError: 142708:0:(acceptor.c:264:lnet_acceptor_remove_socket()) Interface ens2 not found [ 2537.740695] LNet: Removed LNI 192.168.201.140@tcp [ 2538.100359] Key type .llcrypt unregistered [ 2538.102126] Key type ._llcrypt unregistered [ 2549.033174] Key type ._llcrypt registered [ 2549.034694] Key type .llcrypt registered [ 2549.080340] Lustre: DEBUG MARKER: oleg140-server.virtnet: executing load_modules_local [ 2549.431826] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 2549.452551] alg: No test for adler32 (adler32-zlib) [ 2550.335187] Lustre: Lustre: Build Version: 2.16.61_41_g6a6a76f [ 2550.440241] LNet: Added LNI 192.168.201.140@tcp [8/256/0/180] [ 2552.032193] Key type lgssc registered [ 2552.470309] Lustre: Echo OBD driver; http://www.lustre.org/ [ 2555.527501] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 2555.536439] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 2556.740152] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 2558.282416] Lustre: DEBUG MARKER: oleg140-server.virtnet: executing set_default_debug -1 all [ 2561.881321] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 2562.067947] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 2563.736793] Lustre: DEBUG MARKER: oleg140-server.virtnet: executing set_default_debug -1 all [ 2566.175666] Lustre: DEBUG MARKER: oleg140-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 2567.794920] Lustre: DEBUG MARKER: oleg140-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 2570.757167] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 2570.919848] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 2571.947778] Lustre: lustre-OST0000: new connection from lustre-MDT0001-mdtlov (cleaning up unused objects from 0x280000400:72 to 0x280000400:161) [ 2573.331532] Lustre: DEBUG MARKER: oleg140-server.virtnet: executing set_default_debug -1 all [ 2575.559283] Lustre: DEBUG MARKER: oleg140-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 2581.162867] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:170 to 0x280000401:193) [ 2599.529905] Lustre: lustre-OST0000: evicting dd4f8d8c-4939-4143-a717-00bc56b073c2 (at 192.168.201.40@tcp) by administrative request [ 2599.537581] Lustre: Skipped 1 previous similar message [ 2627.555637] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 2627.556597] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2627.560457] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 2627.566778] Lustre: Skipped 1 previous similar message [ 2630.262118] LustreError: 146883:0:(obd_class.h:479:obd_check_dev()) Device 21 not setup [ 2630.310127] Lustre: server umount lustre-OST0000 complete [ 2632.674730] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2632.675712] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 2632.679934] Lustre: Skipped 1 previous similar message [ 2632.684846] Lustre: Skipped 2 previous similar messages [ 2637.793325] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 2637.796690] Lustre: Skipped 1 previous similar message [ 2638.084529] LustreError: 147086:0:(obd_class.h:479:obd_check_dev()) Device 9 not setup [ 2638.087194] LustreError: 147086:0:(obd_class.h:479:obd_check_dev()) Skipped 3 previous similar messages [ 2638.132187] Lustre: server umount lustre-MDT0000 complete [ 2639.827411] LustreError: 144059:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1763354224 with bad export cookie 555003507976461742 [ 2639.827960] LustreError: MGC192.168.201.140@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 2639.833125] LustreError: 144059:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) Skipped 3 previous similar messages [ 2639.925148] LustreError: 147288:0:(obd_class.h:479:obd_check_dev()) Device 16 not setup [ 2639.928018] LustreError: 147288:0:(obd_class.h:479:obd_check_dev()) Skipped 5 previous similar messages [ 2639.989727] Lustre: server umount lustre-MDT0001 complete [ 2644.157179] Lustre: DEBUG MARKER: oleg140-server.virtnet: executing unload_modules_local [ 2645.417314] Key type lgssc unregistered [ 2645.567217] LNet: 147822:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 2645.572142] LNetError: 147822:0:(acceptor.c:264:lnet_acceptor_remove_socket()) Interface ens2 not found [ 2645.580290] LNet: Removed LNI 192.168.201.140@tcp [ 2645.943132] Key type .llcrypt unregistered [ 2645.944629] Key type ._llcrypt unregistered [ 2675.940377] Key type ._llcrypt registered [ 2675.941906] Key type .llcrypt registered [ 2675.987187] Lustre: DEBUG MARKER: oleg140-server.virtnet: executing load_modules_local [ 2676.347748] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 2676.356156] alg: No test for adler32 (adler32-zlib) [ 2677.233874] Lustre: Lustre: Build Version: 2.16.61_41_g6a6a76f [ 2677.333180] LNet: Added LNI 192.168.201.140@tcp [8/256/0/180] [ 2678.920198] Key type lgssc registered [ 2679.369795] Lustre: Echo OBD driver; http://www.lustre.org/ [ 2682.781108] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 2682.794146] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 2684.003353] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 2685.557139] Lustre: DEBUG MARKER: oleg140-server.virtnet: executing set_default_debug -1 all [ 2689.150310] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 2689.324915] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 2690.880408] Lustre: DEBUG MARKER: oleg140-server.virtnet: executing set_default_debug -1 all [ 2693.207212] Lustre: DEBUG MARKER: oleg140-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 2694.704389] Lustre: DEBUG MARKER: oleg140-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 2697.552358] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 2697.718210] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 2699.754566] Lustre: lustre-OST0000: new connection from lustre-MDT0001-mdtlov (cleaning up unused objects from 0x280000400:72 to 0x280000400:193) [ 2700.040450] Lustre: DEBUG MARKER: oleg140-server.virtnet: executing set_default_debug -1 all [ 2702.359128] Lustre: DEBUG MARKER: oleg140-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 2708.966755] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:195 to 0x280000401:225) [ 2719.202673] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 2719.204444] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2719.205188] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 2719.213092] Lustre: Skipped 1 previous similar message [ 2722.292087] LustreError: 152540:0:(obd_class.h:479:obd_check_dev()) Device 21 not setup [ 2722.347123] Lustre: server umount lustre-OST0000 complete [ 2724.324088] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2724.325037] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 2724.329706] Lustre: Skipped 1 previous similar message [ 2724.336867] Lustre: Skipped 1 previous similar message [ 2729.441818] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 2729.445138] Lustre: Skipped 1 previous similar message [ 2730.055057] LustreError: 152742:0:(obd_class.h:479:obd_check_dev()) Device 9 not setup [ 2730.057710] LustreError: 152742:0:(obd_class.h:479:obd_check_dev()) Skipped 3 previous similar messages [ 2730.107639] Lustre: server umount lustre-MDT0000 complete [ 2731.823328] LustreError: 150246:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1763354316 with bad export cookie 8410298736946327329 [ 2731.825226] LustreError: MGC192.168.201.140@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 2731.829404] LustreError: 150246:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 2731.925710] LustreError: 152943:0:(obd_class.h:479:obd_check_dev()) Device 16 not setup [ 2731.928474] LustreError: 152943:0:(obd_class.h:479:obd_check_dev()) Skipped 5 previous similar messages [ 2732.008877] Lustre: server umount lustre-MDT0001 complete [ 2738.024762] Lustre: DEBUG MARKER: oleg140-server.virtnet: executing unload_modules_local [ 2739.302537] Key type lgssc unregistered [ 2739.455502] LNet: 153476:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 2739.460322] LNetError: 153476:0:(acceptor.c:264:lnet_acceptor_remove_socket()) Interface ens2 not found [ 2739.470183] LNet: Removed LNI 192.168.201.140@tcp [ 2739.831479] Key type .llcrypt unregistered [ 2739.833110] Key type ._llcrypt unregistered [ 2751.239237] Key type ._llcrypt registered [ 2751.240613] Key type .llcrypt registered [ 2751.286558] Lustre: DEBUG MARKER: oleg140-server.virtnet: executing load_modules_local [ 2751.634923] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 2751.660915] alg: No test for adler32 (adler32-zlib) [ 2752.562116] Lustre: Lustre: Build Version: 2.16.61_41_g6a6a76f [ 2752.680988] LNet: Added LNI 192.168.201.140@tcp [8/256/0/180] [ 2754.288109] Key type lgssc registered [ 2754.762444] Lustre: Echo OBD driver; http://www.lustre.org/ [ 2758.285317] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 2761.900798] Lustre: DEBUG MARKER: == conf-sanity test 100: check lshowmount lists MGS, MDT, OST and 0@lo ========================================================== 23:39:06 (1763354346) [ 2769.050830] Lustre: DEBUG MARKER: oleg140-server.virtnet: executing set_hostid [ 2773.269198] Lustre: DEBUG MARKER: oleg140-server.virtnet: executing load_modules_local [ 2777.737530] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 2780.630828] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 2783.318634] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 2786.014243] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 2789.160541] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 2789.193780] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 2789.200120] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 2790.300819] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 2790.316771] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 2790.354623] Lustre: lustre-MDT0000: new disk, initializing [ 2790.388779] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 2790.398544] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 2791.902844] Lustre: DEBUG MARKER: oleg140-server.virtnet: executing set_default_debug -1 all [ 2796.908632] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 2796.940582] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 2796.992344] Lustre: 157670:0:(mgs_llog.c:1346:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 2797.028223] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [ 2797.030510] Lustre: Skipped 1 previous similar message [ 2797.072313] Lustre: lustre-MDT0001: new disk, initializing [ 2797.129091] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 2797.149226] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [ 2797.155030] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 2798.845482] Lustre: DEBUG MARKER: oleg140-server.virtnet: executing set_default_debug -1 all [ 2802.522311] Lustre: DEBUG MARKER: oleg140-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 2804.106486] Lustre: DEBUG MARKER: oleg140-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 2807.001861] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 2807.034982] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 2807.150393] Lustre: lustre-OST0000: new disk, initializing [ 2807.152997] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 2807.190518] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 2809.399950] Lustre: DEBUG MARKER: oleg140-server.virtnet: executing set_default_debug -1 all [ 2812.798409] Lustre: DEBUG MARKER: oleg140-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 2815.490042] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [ 2815.494601] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 2815.538879] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 2825.699270] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2825.699596] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 2825.705469] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 2825.707794] LustreError: Skipped 1 previous similar message [ 2825.711615] Lustre: Skipped 1 previous similar message [ 2828.025115] LustreError: 160257:0:(obd_class.h:479:obd_check_dev()) Device 19 not setup [ 2828.070854] Lustre: server umount lustre-OST0000 complete [ 2830.818424] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2830.819320] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 2830.823128] Lustre: Skipped 2 previous similar messages [ 2830.826785] Lustre: Skipped 1 previous similar message [ 2835.522638] LustreError: 160459:0:(obd_class.h:479:obd_check_dev()) Device 22 not setup [ 2835.525232] LustreError: 160459:0:(obd_class.h:479:obd_check_dev()) Skipped 3 previous similar messages [ 2835.557546] Lustre: server umount lustre-MDT0000 complete [ 2837.174317] LustreError: 157661:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1763354422 with bad export cookie 6272247411785375192 [ 2837.178941] LustreError: 157661:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 2837.178951] LustreError: MGC192.168.201.140@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 2837.249111] LustreError: 160660:0:(obd_class.h:479:obd_check_dev()) Device 21 not setup [ 2837.252255] LustreError: 160660:0:(obd_class.h:479:obd_check_dev()) Skipped 5 previous similar messages [ 2837.339812] Lustre: server umount lustre-MDT0001 complete [ 2841.611744] Lustre: DEBUG MARKER: oleg140-server.virtnet: executing unload_modules_local [ 2842.820505] Key type lgssc unregistered [ 2842.979217] LNet: 161191:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 2842.982666] LNetError: 161191:0:(acceptor.c:264:lnet_acceptor_remove_socket()) Interface ens2 not found [ 2842.992628] LNet: Removed LNI 192.168.201.140@tcp [ 2843.332634] Key type .llcrypt unregistered [ 2843.334228] Key type ._llcrypt unregistered [ 2853.743275] Key type ._llcrypt registered [ 2853.744460] Key type .llcrypt registered [ 2853.789911] Lustre: DEBUG MARKER: oleg140-server.virtnet: executing load_modules_local [ 2854.242471] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 2854.253373] alg: No test for adler32 (adler32-zlib) [ 2855.122986] Lustre: Lustre: Build Version: 2.16.61_41_g6a6a76f [ 2855.218603] LNet: Added LNI 192.168.201.140@tcp [8/256/0/180] [ 2856.816199] Key type lgssc registered [ 2857.236742] Lustre: Echo OBD driver; http://www.lustre.org/ [ 2860.174260] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 2860.190153] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 2861.388284] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 2862.984873] Lustre: DEBUG MARKER: oleg140-server.virtnet: executing set_default_debug -1 all [ 2866.676328] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 2866.823684] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 2868.486265] Lustre: DEBUG MARKER: oleg140-server.virtnet: executing set_default_debug -1 all [ 2870.870958] Lustre: DEBUG MARKER: oleg140-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 2872.537813] Lustre: DEBUG MARKER: oleg140-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 2875.531239] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 2875.677511] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 2877.953575] Lustre: DEBUG MARKER: oleg140-server.virtnet: executing set_default_debug -1 all [ 2880.060517] Lustre: DEBUG MARKER: oleg140-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 2885.930921] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:3 to 0x280000401:33) [ 2893.346107] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 2893.354652] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2893.360104] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 2893.363395] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2893.371030] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 2893.760423] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2894.165577] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 2894.168660] Lustre: Skipped 1 previous similar message [ 2894.179469] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2894.185080] Lustre: Skipped 1 previous similar message [ 2894.187809] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 2894.190615] Lustre: Skipped 1 previous similar message [ 2894.192582] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2894.197619] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 2894.200832] Lustre: Skipped 1 previous similar message [ 2894.566322] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2894.903959] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2895.259618] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 2895.263772] Lustre: Skipped 2 previous similar messages [ 2895.270847] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2895.281234] Lustre: Skipped 2 previous similar messages [ 2895.283720] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 2895.286755] Lustre: Skipped 2 previous similar messages [ 2895.288630] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2895.295690] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 2895.298896] Lustre: Skipped 2 previous similar messages [ 2895.641403] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2895.976183] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2896.306937] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2896.646639] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2897.003987] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2897.322989] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 2897.326136] Lustre: Skipped 5 previous similar messages [ 2897.332527] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2897.337656] Lustre: Skipped 5 previous similar messages [ 2897.339678] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 2897.342594] Lustre: Skipped 5 previous similar messages [ 2897.344156] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2897.348420] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 2897.351863] Lustre: Skipped 5 previous similar messages [ 2897.687367] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2898.048753] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2898.401536] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2898.769258] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2899.108470] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2899.455942] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2899.811211] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2900.171712] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2900.550176] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2900.907648] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2901.266926] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2901.599745] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 2901.602986] Lustre: Skipped 11 previous similar messages [ 2901.609964] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2901.615609] Lustre: Skipped 11 previous similar messages [ 2901.617887] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 2901.620614] Lustre: Skipped 11 previous similar messages [ 2901.622917] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2901.627989] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 2901.630700] Lustre: Skipped 11 previous similar messages [ 2902.006968] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2902.364471] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2902.728516] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2903.090777] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2903.431283] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2903.774350] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2904.135956] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2904.498423] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2904.834864] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2905.188715] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2905.538317] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2905.895197] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2906.269281] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2906.648260] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2907.011117] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2907.373874] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2907.755204] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2908.104465] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2908.445715] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2908.782599] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2909.132218] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2909.483166] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2909.835513] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 2909.838789] Lustre: Skipped 22 previous similar messages [ 2909.846751] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2909.851620] Lustre: Skipped 22 previous similar messages [ 2909.853744] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 2909.856466] Lustre: Skipped 22 previous similar messages [ 2909.858521] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2909.863593] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 2909.866663] Lustre: Skipped 22 previous similar messages [ 2910.197249] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2910.530410] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2910.865408] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2911.206270] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2911.553221] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2911.901883] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2912.256149] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2912.582757] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2912.917980] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2913.266648] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2913.609042] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2913.964736] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2914.332367] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2914.720774] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2915.090370] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2915.446235] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2915.804300] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2916.153073] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2916.519781] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2916.873840] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2917.254352] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2917.621063] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2917.983645] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2918.353678] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2918.740720] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2919.133782] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2919.511772] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2919.914542] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2920.311971] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2920.687273] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2921.047108] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2921.387974] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2921.746866] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2922.104631] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2922.459902] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2922.793048] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2923.133227] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2923.490741] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2923.833825] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2924.200797] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2924.564899] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2924.918588] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2925.287084] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2925.674226] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2926.020312] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 2926.022910] Lustre: Skipped 44 previous similar messages [ 2926.029750] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2926.035096] Lustre: Skipped 44 previous similar messages [ 2926.037045] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 2926.039816] Lustre: Skipped 44 previous similar messages [ 2926.041804] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2926.046181] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 2926.048894] Lustre: Skipped 44 previous similar messages [ 2926.411252] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2926.793663] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2927.145835] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2927.490921] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2927.833965] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2928.181978] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2928.571676] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2928.921450] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2929.305012] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2929.654913] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2930.018889] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2930.390263] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2930.722926] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2931.064966] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2931.404899] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2931.754884] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2932.133750] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2932.491274] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2932.846161] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2933.205799] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2933.552850] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2933.910559] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2934.277118] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2934.651233] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2934.999203] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2935.348624] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2935.692197] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2936.047627] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2936.443375] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2936.823875] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2937.198909] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2937.560319] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2937.916026] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2938.284128] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2938.664621] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2939.021822] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2939.416798] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2939.797652] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2940.160039] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2940.503841] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2940.856966] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2941.197742] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2941.537020] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2941.874369] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2942.247202] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2942.581423] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2942.926917] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2943.269618] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2943.639118] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2943.977223] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2944.344387] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2944.680685] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2945.027300] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2945.376848] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2945.732146] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2946.083804] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2946.466752] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2946.838578] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2947.239053] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2947.643605] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2948.060905] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2948.447029] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2948.789530] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2949.208856] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2949.627924] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2950.020166] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2950.408298] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2950.803303] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2951.186954] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2951.530819] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2951.878164] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2952.245303] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2952.634879] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2952.993993] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2953.357316] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2953.715459] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2954.075238] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2954.450689] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2954.851347] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2955.220894] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2955.578868] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2955.956368] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2956.332863] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2956.676996] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2957.014351] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2957.363782] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2957.758460] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2958.109663] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 2958.112383] Lustre: Skipped 87 previous similar messages [ 2958.118470] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2958.123599] Lustre: Skipped 87 previous similar messages [ 2958.126059] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 2958.128838] Lustre: Skipped 87 previous similar messages [ 2958.130897] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2958.136611] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 2958.140674] Lustre: Skipped 87 previous similar messages [ 2958.520575] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2958.864286] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2959.225576] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2959.597973] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2959.977783] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2960.362918] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2960.735847] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2961.128157] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2961.513789] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2961.868170] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2962.256790] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2962.623360] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2962.977507] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2963.374898] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2963.758424] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2964.140141] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2964.520200] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2964.893215] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2965.262709] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2965.629573] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2966.010848] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2966.372271] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2966.740459] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2967.104468] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2967.457318] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2967.821727] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2968.189191] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2968.530083] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2968.908493] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2969.274965] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2969.664120] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2970.029717] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2970.402031] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2970.774251] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2971.171900] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2971.540305] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2971.897104] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2972.251395] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2972.640842] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2973.019122] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2973.460262] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2973.857591] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2974.244941] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2974.621166] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2974.966158] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2975.329692] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2975.696602] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2976.073887] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2976.472162] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2976.881907] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2977.269836] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2977.622268] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2978.018162] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2978.416923] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2978.784780] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2979.184889] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2979.536072] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2979.904916] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2980.280939] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2980.656990] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2981.100319] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2981.524840] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2981.912508] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2982.267767] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2982.662847] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2983.074415] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2983.442970] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2983.801444] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2984.249457] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2984.671253] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2985.114699] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2985.549935] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2985.953549] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2986.419759] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2986.875844] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2987.326400] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2987.778692] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2988.251399] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2988.658312] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2989.051534] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2989.480871] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2989.883144] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2990.251108] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2990.628048] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2991.045257] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2991.405941] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2991.843927] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2992.211780] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2992.619187] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2993.072242] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2993.467689] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2993.864278] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2994.334940] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2994.703588] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2995.130029] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2995.498775] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2995.888587] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2996.286700] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2996.719245] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2997.143814] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2997.574776] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2997.974714] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2998.451963] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2998.903717] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2999.376647] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2999.785098] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3000.186119] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3000.588713] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3000.950186] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3001.343449] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3001.729905] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3002.098336] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3002.464868] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3002.867201] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3003.276187] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3003.625883] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3004.006129] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3004.391994] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3004.750346] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3005.062754] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3005.429825] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3005.792746] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3006.180635] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3006.548892] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3006.990715] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3007.413063] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3007.781608] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3008.200212] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3008.625655] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3009.039436] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3009.563172] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3010.042805] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3010.480246] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3010.907997] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3011.375135] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3011.805337] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3012.266145] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3012.709607] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3013.150042] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3013.608732] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3014.069445] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3014.495360] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3014.918302] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3015.335478] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3015.783024] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3016.197757] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3016.631423] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3017.026531] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3017.425384] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3017.833815] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3018.295101] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3018.762760] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3019.263868] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3019.735308] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3020.214193] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3020.669758] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3021.115045] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3021.539822] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3021.955186] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3022.392581] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 3022.394801] Lustre: Skipped 159 previous similar messages [ 3022.400661] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 3022.405936] Lustre: Skipped 159 previous similar messages [ 3022.407912] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 3022.411091] Lustre: Skipped 159 previous similar messages [ 3022.413243] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3022.417890] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 3022.420653] Lustre: Skipped 159 previous similar messages [ 3022.909740] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3023.351431] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3023.799041] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3024.295681] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3024.769815] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3025.210810] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3025.615120] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3025.994591] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3026.364072] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3026.728789] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3027.115274] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3027.493855] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3027.880506] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3028.286275] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3028.668857] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3029.041822] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3029.442439] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3029.805683] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3030.174682] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3030.542052] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3030.904441] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3031.245050] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3031.596770] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3031.955287] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3032.297595] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3032.634516] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3032.982665] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3033.324954] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3033.657912] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3034.021723] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3034.424588] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3034.792110] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3035.170312] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3035.544731] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3035.935076] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3036.276791] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3036.623715] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3036.973533] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3037.360411] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3037.718582] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3038.093500] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3038.439050] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3038.795679] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3039.168055] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3039.550325] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3039.947798] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3040.359895] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3040.747148] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3041.129852] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3041.464967] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3041.825369] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3042.164381] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3042.513348] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3042.866798] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3043.242348] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3043.588669] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3043.940278] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3044.282838] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3044.655050] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3045.031561] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3045.404995] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3045.781507] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3046.179710] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3046.543439] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3046.914684] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3047.267953] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3047.622320] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3047.992233] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3048.351972] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3048.706327] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3049.078849] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3049.458149] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3049.840513] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3050.234887] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3050.605333] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3050.988193] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3051.370628] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3051.751408] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3052.135087] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3052.496646] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3052.908616] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3053.288964] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3053.655718] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3053.996439] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3054.355962] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3054.718942] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3055.119664] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3055.544101] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3055.933497] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3056.341642] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3056.699514] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3057.081727] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3057.430957] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3057.811041] LustreError: 162585:0:(osp_precreate.c:651:osp_precreate_send()) lustre-OST0000-osc-MDT0000: can't precreate: rc = -5 [ 3057.816942] LustreError: 162585:0:(osp_precreate.c:1355:osp_precreate_thread()) lustre-OST0000-osc-MDT0000: cannot precreate objects: rc = -5 [ 3057.817424] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3058.267172] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3058.667486] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3059.063763] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3059.481773] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3059.907967] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3060.359607] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3060.835323] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3061.360173] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3061.784498] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3062.175693] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3062.546589] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3062.960452] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3063.371120] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3063.736818] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3064.149521] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3064.525788] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3064.888973] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3065.249160] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3065.616654] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3065.978742] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3066.338799] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3066.662318] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3067.016236] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3067.379986] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3067.767497] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3068.116310] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3068.473906] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3068.832980] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3069.196727] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3069.573945] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3069.958616] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3070.395575] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3070.835868] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3071.269371] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3071.633282] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3072.011673] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3072.383466] LustreError: 162585:0:(osp_precreate.c:651:osp_precreate_send()) lustre-OST0000-osc-MDT0000: can't precreate: rc = -5 [ 3072.389430] LustreError: 162585:0:(osp_precreate.c:1355:osp_precreate_thread()) lustre-OST0000-osc-MDT0000: cannot precreate objects: rc = -5 [ 3072.390578] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3072.754751] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3073.089842] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3073.424681] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3073.775898] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3074.109274] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3074.475052] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3074.836201] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3075.198742] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3075.562169] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3075.935250] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3076.322590] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3076.743042] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3077.115367] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3077.491111] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3077.846192] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3078.223762] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3078.577645] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3078.892916] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3079.246224] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3079.581561] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3079.890922] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3080.227573] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3080.572595] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3080.926360] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3081.264780] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3081.657408] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3082.030318] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3082.408634] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3082.776322] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3083.121717] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3083.468039] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3083.807646] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3084.154471] LustreError: 162585:0:(osp_precreate.c:651:osp_precreate_send()) lustre-OST0000-osc-MDT0000: can't precreate: rc = -5 [ 3084.158252] LustreError: 162585:0:(osp_precreate.c:1355:osp_precreate_thread()) lustre-OST0000-osc-MDT0000: cannot precreate objects: rc = -5 [ 3084.159546] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3084.503975] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3084.892430] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3085.269810] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3085.621976] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3086.018150] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3086.377966] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3086.764135] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3087.114082] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3087.489207] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3087.832300] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3088.178566] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3088.509131] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3088.830682] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3089.175268] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3089.516910] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3089.860666] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3090.215644] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3090.549862] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3090.951343] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3091.351167] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3091.756704] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3092.176313] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3092.570209] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3092.945666] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3093.323531] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3093.748567] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3094.147669] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3094.543234] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3094.976072] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3095.370817] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3095.766207] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3096.192047] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3096.596820] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3096.965813] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3097.302938] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3097.642654] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3098.003559] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3098.350833] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3098.688876] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3099.033814] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3099.378091] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3099.741462] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3100.161304] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3100.511446] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3100.866711] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3101.252500] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3101.639152] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3102.004932] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3102.371431] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3102.739745] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3103.103037] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3103.465331] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3103.831123] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3104.185528] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3104.544158] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3104.886128] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3105.231032] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3105.568063] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3105.926978] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3106.306185] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3106.661633] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3107.031293] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3107.350093] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3107.697087] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3108.039357] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3108.379197] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3108.718917] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3109.056674] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3109.407905] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3109.757692] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3110.091831] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3110.431055] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3110.781942] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3111.140371] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3111.497686] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3111.844640] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3112.211374] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3112.567120] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3112.921510] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3113.247667] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3113.578886] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3113.902747] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3114.231277] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3114.584565] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3114.930129] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3115.279460] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3115.673234] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3116.049799] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3116.425896] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3116.781891] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3117.123896] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3117.472329] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3117.841220] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3118.206671] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3118.576734] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3118.941360] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3119.318282] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3119.713311] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3120.093418] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3120.458273] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3120.834282] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3121.181518] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3121.543213] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3121.899964] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3122.262623] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3122.605581] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3122.978119] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3123.339038] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3123.697370] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3124.071646] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3124.411261] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3124.759077] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3125.090103] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3125.441413] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3125.774925] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3126.111304] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3126.467979] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3126.805827] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3127.159662] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3127.501788] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3127.840541] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3128.187567] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3128.528320] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3128.872040] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3129.218030] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3129.594207] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3129.961511] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3130.310589] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3130.649926] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3131.014284] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3131.386881] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3131.738613] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3132.074142] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3132.440176] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3132.782645] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3133.134487] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3133.472470] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3133.808701] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3134.181665] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3134.583293] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3134.976497] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3135.327576] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3135.698689] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3136.029595] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3136.371608] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3136.728533] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3137.070138] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3137.412556] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3137.773309] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3138.118577] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3138.454839] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3138.794648] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3139.143851] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3139.483906] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3139.865371] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3140.255605] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3140.631148] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3141.011120] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3141.381962] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3141.765672] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3142.171531] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3142.548541] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3142.892061] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3143.247507] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3143.594917] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3143.971700] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3144.336297] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3144.721170] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3145.106297] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3145.482240] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3145.890502] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3146.251890] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3146.622435] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3147.001778] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3147.384638] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3147.745655] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3148.124636] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3148.489735] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3148.866891] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3149.232069] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3149.636489] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3150.045989] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3150.413854] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 3150.416661] Lustre: Skipped 346 previous similar messages [ 3150.423772] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 3150.429287] Lustre: Skipped 346 previous similar messages [ 3150.431605] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 3150.434642] Lustre: Skipped 346 previous similar messages [ 3150.436992] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3150.443850] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 3150.446698] Lustre: Skipped 346 previous similar messages [ 3150.807758] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3151.192521] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3151.571753] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3151.987817] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3152.346338] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3152.727688] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3153.108569] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3153.471950] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3153.822612] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3154.181451] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3154.567194] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3155.000625] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3155.364195] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3155.720193] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3156.078812] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3156.426385] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3156.784716] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3157.136941] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3157.496064] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3157.844162] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3158.196694] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3158.533190] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3158.878067] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3159.226264] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3159.609624] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3160.020715] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3160.382316] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3160.759081] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3161.112251] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3161.464267] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3161.800253] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3162.144504] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3162.504660] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3162.871443] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3163.217783] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3163.568042] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3163.925248] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3164.292425] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3164.675286] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3165.064898] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3165.404167] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3165.756125] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3166.119547] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3166.490297] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3166.826608] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3167.195976] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3167.541860] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3167.894879] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3168.259590] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3168.624983] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3168.979155] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3169.344125] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3169.759729] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3170.171889] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3170.530562] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3170.870611] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3171.249754] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3171.604943] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3171.959978] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3172.311036] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3172.683359] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3173.050754] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3173.427099] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3173.778244] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3174.130633] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3174.504198] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3174.868909] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3175.213276] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3175.572169] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3175.924030] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3176.318942] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3176.706434] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3177.068609] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3177.456966] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3177.839187] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3178.221469] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3178.598626] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3178.966879] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3179.339360] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3179.770669] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3180.171473] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3180.551330] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3180.922058] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3181.300071] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3181.688453] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3182.058311] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3182.410411] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3182.800564] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3183.159109] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3183.496586] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3183.845394] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3184.211928] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3184.618036] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3185.071713] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3185.481438] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3185.868777] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3186.248712] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3186.680590] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3187.073380] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3187.458037] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3187.853087] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3188.268547] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3188.659524] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3189.077318] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3189.469065] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3189.897060] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3190.245687] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3190.600601] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3190.980768] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3191.336125] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3191.666723] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3191.996691] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3192.342898] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3192.713437] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3193.079299] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3193.453742] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3193.798973] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3194.138314] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3194.492269] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3194.832935] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3195.156793] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3195.508490] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3195.878863] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3196.203684] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3196.535280] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3196.891144] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3197.242388] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3197.590848] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3197.965082] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3198.353732] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3198.748471] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3199.156314] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3199.516934] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3199.914886] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3200.279770] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3200.643611] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3200.994675] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3201.405094] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3201.789768] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3202.130809] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3202.489967] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3202.843623] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3203.174839] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3203.591126] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3203.967260] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3204.346180] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3204.740587] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3205.096314] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3205.467038] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3205.824758] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3206.194256] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3206.559645] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3206.939685] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3207.320057] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3207.665149] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3208.033984] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3208.394991] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3208.760966] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3209.099539] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3209.465388] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3209.811936] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3210.152657] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3210.490244] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3210.842860] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3211.178318] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3211.529654] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3211.861160] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3212.193347] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3212.523235] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3212.854763] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3213.166274] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3213.471455] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3213.803811] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3214.159040] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3214.502806] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3214.851173] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3215.219345] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3215.548195] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3215.877803] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3216.205757] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3216.540135] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3216.862269] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3217.184544] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3217.526104] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3217.861865] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3218.186465] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3218.507171] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3218.862892] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3219.213559] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3219.559154] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3219.941819] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3220.275227] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3220.600757] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3220.938288] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3221.272053] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3221.606424] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3221.927483] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3222.249634] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3222.562584] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3222.874753] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3223.189694] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3223.500178] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3223.817641] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3224.159192] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3224.481816] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3224.831077] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3225.175346] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3225.515406] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3225.845975] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3226.185933] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3226.516381] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3226.852959] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3227.181494] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3227.515429] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3227.856579] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3228.204389] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3228.534519] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3228.884241] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3229.263306] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3229.650283] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3230.029199] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3230.445300] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3230.785730] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3231.129979] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3231.465256] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3231.835670] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3232.174848] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3232.511932] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3232.843455] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3233.182424] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3233.518077] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3233.861089] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3234.220042] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3234.615743] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3234.970159] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3235.331340] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3235.682930] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3236.020833] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3236.372601] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3236.699941] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3237.046413] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3237.424533] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3237.785918] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3238.146691] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3238.492920] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3238.849164] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3239.211672] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3239.598756] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3240.006348] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3240.403191] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3240.742159] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3241.051145] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3241.359959] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3241.700763] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3242.028876] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3242.366765] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3242.691131] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3243.013257] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3243.335674] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3243.693093] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3244.091913] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3244.485341] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3244.851758] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3245.211292] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3245.528540] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3245.863831] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3246.184157] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3246.512408] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3246.832623] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3247.160906] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3247.488618] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3247.828502] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3248.140846] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3248.459430] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3248.772785] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3249.108772] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3249.465126] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3249.822177] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3250.195490] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3250.555379] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3250.894958] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3251.208503] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3251.516483] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3251.832513] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3252.145289] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3252.494117] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3252.824923] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3253.148535] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3253.486851] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3253.829575] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3254.151472] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3254.491383] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3254.824473] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3255.166545] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3255.532775] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3255.864133] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3256.185215] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3256.514019] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3256.825707] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3257.133223] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3257.437324] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3257.770175] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3258.088447] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3258.428099] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3258.755326] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3259.127140] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3259.456105] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3259.757147] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3260.060229] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3260.384778] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3260.689137] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3260.983542] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3261.277648] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3410.678816] LustreError: 164163:0:(ofd_dev.c:1780:ofd_create_hdl()) lustre-OST0000: unable to precreate: rc = -28 [ 3410.682647] LustreError: 162585:0:(osp_precreate.c:651:osp_precreate_send()) lustre-OST0000-osc-MDT0000: can't precreate: rc = -28 [ 3522.016849] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 3522.020871] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 3522.026858] Lustre: Skipped 313 previous similar messages [ 3522.029702] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 3523.041385] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 3523.045229] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 3526.114573] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 3527.350453] LustreError: 216337:0:(obd_class.h:479:obd_check_dev()) Device 21 not setup [ 3527.419829] Lustre: server umount lustre-OST0000 complete [ 3531.234758] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 3531.237242] Lustre: Skipped 2 previous similar messages [ 3534.917087] LustreError: 216539:0:(obd_class.h:479:obd_check_dev()) Device 9 not setup [ 3534.919834] LustreError: 216539:0:(obd_class.h:479:obd_check_dev()) Skipped 3 previous similar messages [ 3535.014722] Lustre: server umount lustre-MDT0000 complete [ 3536.353098] LustreError: 162560:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 3536.360297] LustreError: 162560:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 3 previous similar messages [ 3536.463132] LustreError: 162547:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1763355121 with bad export cookie 8953834730254989249 [ 3536.466414] LustreError: MGC192.168.201.140@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 3536.467666] LustreError: 162547:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 3536.536506] LustreError: 216741:0:(obd_class.h:479:obd_check_dev()) Device 16 not setup [ 3536.539397] LustreError: 216741:0:(obd_class.h:479:obd_check_dev()) Skipped 5 previous similar messages [ 3536.604750] Lustre: server umount lustre-MDT0001 complete [ 3540.753989] Lustre: DEBUG MARKER: oleg140-server.virtnet: executing unload_modules_local [ 3541.839979] Key type lgssc unregistered [ 3541.976397] LNet: 217272:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 3541.979304] LNetError: 217272:0:(acceptor.c:264:lnet_acceptor_remove_socket()) Interface ens2 not found [ 3541.991399] LNet: Removed LNI 192.168.201.140@tcp [ 3542.336217] Key type .llcrypt unregistered [ 3542.337835] Key type ._llcrypt unregistered [ 3552.342532] Key type ._llcrypt registered [ 3552.344078] Key type .llcrypt registered [ 3552.388691] Lustre: DEBUG MARKER: oleg140-server.virtnet: executing load_modules_local [ 3552.731431] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 3552.739618] alg: No test for adler32 (adler32-zlib) [ 3553.606466] Lustre: Lustre: Build Version: 2.16.61_41_g6a6a76f [ 3553.694254] LNet: Added LNI 192.168.201.140@tcp [8/256/0/180] [ 3555.280145] Key type lgssc registered [ 3555.677895] Lustre: Echo OBD driver; http://www.lustre.org/ [ 3558.562431] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 3558.572396] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 3559.787301] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 3561.145429] Lustre: DEBUG MARKER: oleg140-server.virtnet: executing set_default_debug -1 all [ 3564.291357] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 3564.447788] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 3565.859038] Lustre: DEBUG MARKER: oleg140-server.virtnet: executing set_default_debug -1 all [ 3567.878680] Lustre: DEBUG MARKER: oleg140-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 3569.291173] Lustre: DEBUG MARKER: oleg140-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 3571.746195] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 3571.873759] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 3573.908914] Lustre: DEBUG MARKER: oleg140-server.virtnet: executing set_default_debug -1 all [ 3575.872853] Lustre: DEBUG MARKER: oleg140-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 3584.170919] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:49666 to 0x280000401:49697) [ 3588.414541] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 3589.044455] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 3589.050643] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 3589.053093] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3589.056186] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 3589.056211] LustreError: 218664:0:(osp_precreate.c:1263:osp_precreate_thread()) cfs_fail_timeout id 2107 sleeping for 20000ms [ 3591.622704] Lustre: lustre-OST0000: Not available for connect from 192.168.201.40@tcp (stopping) [ 3594.723297] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 3594.723620] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 3594.726691] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 3594.730578] Lustre: Skipped 1 previous similar message [ 3594.733643] Lustre: Skipped 1 previous similar message [ 3595.822431] LustreError: 221065:0:(obd_class.h:479:obd_check_dev()) Device 21 not setup [ 3595.860324] Lustre: server umount lustre-OST0000 complete [ 3609.136126] LustreError: 218664:0:(osp_precreate.c:1263:osp_precreate_thread()) cfs_fail_timeout id 2107 awake [ 3624.056276] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 3624.170965] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 3626.029729] Lustre: DEBUG MARKER: oleg140-server.virtnet: executing set_default_debug -1 all [ 3627.999675] Lustre: DEBUG MARKER: oleg140-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 3632.689317] Lustre: DEBUG MARKER: osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid in FULL state after 4 sec [ 3634.004212] Lustre: DEBUG MARKER: oleg140-client.virtnet: executing wait_import_state (FULL|IDLE) osc.lustre-OST0000-osc-ffff926945a8e800.ost_server_uuid 50 [ 3634.527431] Lustre: DEBUG MARKER: osc.lustre-OST0000-osc-ffff926945a8e800.ost_server_uuid in FULL state after 0 sec [ 3635.432847] LustreError: lustre-OST0000-osc-MDT0001: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3635.436708] Lustre: lustre-OST0000-osc-MDT0001: Connection restored to 0@lo (at 0@lo) [ 3635.436839] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3640.802974] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 3640.804369] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 3640.804720] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 3640.806922] Lustre: Skipped 1 previous similar message [ 3641.645080] LustreError: 222386:0:(obd_class.h:479:obd_check_dev()) Device 21 not setup [ 3641.646695] LustreError: 222386:0:(obd_class.h:479:obd_check_dev()) Skipped 3 previous similar messages [ 3641.674404] Lustre: server umount lustre-OST0000 complete [ 3645.921647] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 3645.926888] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 3645.929796] Lustre: Skipped 2 previous similar messages [ 3651.041396] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 3651.043406] Lustre: Skipped 1 previous similar message [ 3657.184178] Lustre: lustre-MDT0000 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 3657.221518] LustreError: 222587:0:(obd_class.h:479:obd_check_dev()) Device 9 not setup [ 3657.224206] LustreError: 222587:0:(obd_class.h:479:obd_check_dev()) Skipped 3 previous similar messages [ 3657.255690] Lustre: server umount lustre-MDT0000 complete [ 3658.551635] LustreError: 218625:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1763355243 with bad export cookie 4045914095294132386 [ 3658.554068] LustreError: MGC192.168.201.140@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 3658.556268] LustreError: 218625:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 3658.680368] Lustre: server umount lustre-MDT0001 complete [ 3662.408549] Lustre: DEBUG MARKER: oleg140-server.virtnet: executing unload_modules_local [ 3663.546900] Key type lgssc unregistered [ 3663.710611] LNet: 223321:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 3663.714175] LNetError: 223321:0:(acceptor.c:264:lnet_acceptor_remove_socket()) Interface ens2 not found [ 3663.723332] LNet: Removed LNI 192.168.201.140@tcp [ 3664.087463] Key type .llcrypt unregistered [ 3664.088281] Key type ._llcrypt unregistered [ 3676.243274] Key type ._llcrypt registered [ 3676.244691] Key type .llcrypt registered [ 3676.288093] Lustre: DEBUG MARKER: oleg140-server.virtnet: executing load_modules_local [ 3676.613045] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 3676.631047] alg: No test for adler32 (adler32-zlib) [ 3677.486793] Lustre: Lustre: Build Version: 2.16.61_41_g6a6a76f [ 3677.578528] LNet: Added LNI 192.168.201.140@tcp [8/256/0/180] [ 3679.168121] Key type lgssc registered [ 3679.570708] Lustre: Echo OBD driver; http://www.lustre.org/ [ 3682.879904] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 3682.886822] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 3684.054751] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 3685.388456] Lustre: DEBUG MARKER: oleg140-server.virtnet: executing set_default_debug -1 all [ 3688.223292] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 3688.344612] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 3689.653714] Lustre: DEBUG MARKER: oleg140-server.virtnet: executing set_default_debug -1 all [ 3690.707600] Lustre: Modifying parameter general.debug_raw_pointers=Y in log params [ 3693.002892] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 3693.132150] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 3694.185949] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:49699 to 0x280000401:49729) [ 3695.107735] Lustre: DEBUG MARKER: oleg140-server.virtnet: executing set_default_debug -1 all [ 3698.102711] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 3698.130648] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 3698.197308] Lustre: lustre-OST0001: new disk, initializing [ 3698.199420] Lustre: srv-lustre-OST0001: No data found on store. Initialize space. [ 3698.230399] Lustre: lustre-OST0001: Imperative Recovery not enabled, recovery window 60-180 [ 3700.136179] Lustre: DEBUG MARKER: oleg140-server.virtnet: executing set_default_debug -1 all [ 3705.629791] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 3707.597275] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x00000002c0000400-0x0000000300000400]:1:ost [ 3707.601301] Lustre: cli-lustre-OST0001-super: Allocated super-sequence [0x00000002c0000400-0x0000000300000400]:1:ost] [ 3707.635984] Lustre: lustre-OST0001-osc-MDT0000: update sequence from 0x100010000 to 0x2c0000401 [ 3710.381589] Lustre: Setting parameter general.lod.*.mdt_hash=crush in log params [ 3724.257021] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 3724.259935] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 3724.264197] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 3728.356265] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 3728.357909] Lustre: lustre-MDT0000-lwp-OST0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 3728.362768] Lustre: Skipped 2 previous similar messages [ 3730.113034] LustreError: 228357:0:(obd_class.h:479:obd_check_dev()) Device 28 not setup [ 3730.148631] Lustre: server umount lustre-MDT0000 complete [ 3731.430481] LustreError: 224985:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1763355316 with bad export cookie 7946558116613119557 [ 3731.434117] LustreError: 225727:0:(osp_object.c:617:osp_attr_get()) lustre-MDT0000-osp-MDT0001: osp_attr_get update error [0x200001b72:0x1:0x0]: rc = -5 [ 3731.435301] LustreError: MGC192.168.201.140@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 3731.435583] LustreError: 224985:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) Skipped 3 previous similar messages [ 3731.501942] LustreError: 228559:0:(obd_class.h:479:obd_check_dev()) Device 27 not setup [ 3731.504458] LustreError: 228559:0:(obd_class.h:479:obd_check_dev()) Skipped 7 previous similar messages [ 3731.558617] Lustre: server umount lustre-MDT0001 complete [ 3743.667118] LustreError: 228761:0:(obd_class.h:479:obd_check_dev()) Device 21 not setup [ 3743.670357] LustreError: 228761:0:(obd_class.h:479:obd_check_dev()) Skipped 7 previous similar messages [ 3743.690853] Lustre: server umount lustre-OST0000 complete [ 3755.251137] LustreError: 228963:0:(obd_class.h:479:obd_check_dev()) Device 2 not setup [ 3755.253007] LustreError: 228963:0:(obd_class.h:479:obd_check_dev()) Skipped 3 previous similar messages [ 3755.297047] Lustre: server umount lustre-OST0001 complete [ 3756.379695] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro [ 3756.417549] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 3756.775082] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro [ 3756.812300] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 3757.167363] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: ,errors=remount-ro [ 3757.195409] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 3757.598632] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: ,errors=remount-ro [ 3757.628880] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 3762.193452] Lustre: DEBUG MARKER: oleg140-server.virtnet: executing load_modules_local [ 3765.692978] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 3765.840375] LustreError: 230370:0:(ldlm_lib.c:1179:target_handle_connect()) mylustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 3765.846926] LustreError: 230370:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 2 previous similar messages [ 3765.861150] Lustre: mylustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 3767.116248] Lustre: DEBUG MARKER: oleg140-server.virtnet: executing set_default_debug -1 all [ 3769.944788] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 3771.428109] Lustre: DEBUG MARKER: oleg140-server.virtnet: executing set_default_debug -1 all [ 3772.414691] Lustre: 231479:0:(mgs_llog.c:1346:mgs_modify_param()) MGS: modify general/debug_raw_pointers=Y (mode = 0) failed: rc = -17 [ 3774.735323] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 3774.852739] Lustre: mylustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 3774.854983] Lustre: Skipped 1 previous similar message [ 3775.908640] LustreError: 231835:0:(ldlm_lib.c:1179:target_handle_connect()) mylustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 3775.912240] Lustre: mylustre-OST0000: new connection from mylustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:49699 to 0x280000401:49761) [ 3776.812056] Lustre: DEBUG MARKER: oleg140-server.virtnet: executing set_default_debug -1 all [ 3779.999206] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 3780.006960] LustreError: 231835:0:(ldlm_lib.c:1179:target_handle_connect()) mylustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 3780.013439] Lustre: mylustre-OST0000: new connection from mylustre-MDT0001-mdtlov (cleaning up unused objects from 0x280000400:100 to 0x280000400:129) [ 3780.015301] LustreError: 231835:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 1 previous similar message [ 3782.159812] Lustre: DEBUG MARKER: oleg140-server.virtnet: executing set_default_debug -1 all [ 3785.282674] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 3786.498712] Lustre: 233327:0:(mgs_llog.c:1346:mgs_modify_param()) MGS: modify general/lod.*.mdt_hash=crush (mode = 0) failed: rc = -17 [ 3795.936897] LustreError: mylustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 3795.940602] Lustre: mylustre-MDT0000-osp-MDT0001: Connection to mylustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 3795.946083] Lustre: mylustre-MDT0000: Not available for connect from 0@lo (stopping) [ 3795.948280] Lustre: Skipped 3 previous similar messages [ 3800.549217] Lustre: mylustre-MDT0000-lwp-OST0001: Connection to mylustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 3800.551560] Lustre: mylustre-MDT0000: Not available for connect from 0@lo (stopping) [ 3800.554093] Lustre: Skipped 1 previous similar message [ 3800.556081] Lustre: Skipped 1 previous similar message [ 3802.043069] LustreError: 233633:0:(obd_class.h:479:obd_check_dev()) Device 10 not setup [ 3802.045049] LustreError: 233633:0:(obd_class.h:479:obd_check_dev()) Skipped 5 previous similar messages [ 3802.071389] Lustre: server umount mylustre-MDT0000 complete [ 3803.273609] LustreError: 230344:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1763355388 with bad export cookie 7946558116613122308 [ 3803.277660] LustreError: MGC192.168.201.140@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 3803.395240] Lustre: server umount mylustre-MDT0001 complete [ 3814.837150] LustreError: 234034:0:(obd_class.h:479:obd_check_dev()) Device 23 not setup [ 3814.839807] LustreError: 234034:0:(obd_class.h:479:obd_check_dev()) Skipped 15 previous similar messages [ 3814.859213] Lustre: server umount mylustre-OST0000 complete [ 3826.330154] Lustre: server umount mylustre-OST0001 complete [ 3827.323184] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro [ 3827.354349] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 3827.700568] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro [ 3827.732203] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 3828.087655] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: ,errors=remount-ro [ 3828.110744] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 3828.468959] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: ,errors=remount-ro [ 3828.509969] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 3832.976526] Lustre: DEBUG MARKER: oleg140-server.virtnet: executing load_modules_local [ 3836.335968] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 3836.494811] LustreError: 235645:0:(ldlm_lib.c:1179:target_handle_connect()) tfs-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 3836.524608] Lustre: tfs-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 3836.527364] Lustre: Skipped 1 previous similar message [ 3837.854209] Lustre: DEBUG MARKER: oleg140-server.virtnet: executing set_default_debug -1 all [ 3840.753922] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 3842.171135] Lustre: DEBUG MARKER: oleg140-server.virtnet: executing set_default_debug -1 all [ 3843.146701] Lustre: 236754:0:(mgs_llog.c:1346:mgs_modify_param()) MGS: modify general/debug_raw_pointers=Y (mode = 0) failed: rc = -17 [ 3845.426544] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 3846.565322] LustreError: 237111:0:(ldlm_lib.c:1179:target_handle_connect()) tfs-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 3846.567898] Lustre: tfs-OST0000: new connection from tfs-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:49699 to 0x280000401:49793) [ 3847.408288] Lustre: DEBUG MARKER: oleg140-server.virtnet: executing set_default_debug -1 all [ 3850.233821] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 3851.375894] Lustre: tfs-OST0000: new connection from tfs-MDT0001-mdtlov (cleaning up unused objects from 0x280000400:131 to 0x280000400:161) [ 3852.341620] Lustre: DEBUG MARKER: oleg140-server.virtnet: executing set_default_debug -1 all [ 3855.533417] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 3856.837120] Lustre: 238602:0:(mgs_llog.c:1346:mgs_modify_param()) MGS: modify general/lod.*.mdt_hash=crush (mode = 0) failed: rc = -17 [ 3866.593043] LustreError: tfs-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 3866.594984] Lustre: tfs-MDT0000-osp-MDT0001: Connection to tfs-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 3866.598051] Lustre: Skipped 1 previous similar message [ 3866.600314] Lustre: tfs-MDT0000: Not available for connect from 0@lo (stopping) [ 3866.602744] Lustre: Skipped 2 previous similar messages [ 3870.017340] LustreError: 238851:0:(obd_class.h:479:obd_check_dev()) Device 10 not setup [ 3870.020401] LustreError: 238851:0:(obd_class.h:479:obd_check_dev()) Skipped 9 previous similar messages [ 3870.051109] Lustre: server umount tfs-MDT0000 complete [ 3871.264553] LustreError: 238604:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1763355456 with bad export cookie 7946558116613124261 [ 3871.268465] LustreError: 238604:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) Skipped 8 previous similar messages [ 3871.269244] LustreError: MGC192.168.201.140@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 3871.401779] Lustre: server umount tfs-MDT0001 complete [ 3881.988657] Lustre: server umount tfs-OST0000 complete [ 3893.535019] Lustre: server umount tfs-OST0001 complete [ 3894.568720] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro [ 3894.595703] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 3894.939622] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro [ 3894.969919] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 3895.335153] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: ,errors=remount-ro [ 3895.357975] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 3895.706306] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: ,errors=remount-ro [ 3895.744202] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 3900.323941] Lustre: DEBUG MARKER: oleg140-server.virtnet: executing load_modules_local [ 3903.956488] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 3904.137885] LustreError: 240861:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 3904.169158] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 3904.171406] Lustre: Skipped 3 previous similar messages [ 3905.469919] Lustre: DEBUG MARKER: oleg140-server.virtnet: executing set_default_debug -1 all [ 3908.284626] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 3909.678301] Lustre: DEBUG MARKER: oleg140-server.virtnet: executing set_default_debug -1 all [ 3910.629660] Lustre: 241972:0:(mgs_llog.c:1346:mgs_modify_param()) MGS: modify general/debug_raw_pointers=Y (mode = 0) failed: rc = -17 [ 3912.729442] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 3913.894950] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:49699 to 0x280000401:49825) [ 3914.664582] Lustre: DEBUG MARKER: oleg140-server.virtnet: executing set_default_debug -1 all [ 3917.502019] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 3918.634972] Lustre: lustre-OST0000: new connection from lustre-MDT0001-mdtlov (cleaning up unused objects from 0x280000400:131 to 0x280000400:193) [ 3919.395357] Lustre: DEBUG MARKER: oleg140-server.virtnet: executing set_default_debug -1 all [ 3922.438198] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 3923.567620] Lustre: 243817:0:(mgs_llog.c:1346:mgs_modify_param()) MGS: modify general/lod.*.mdt_hash=crush (mode = 0) failed: rc = -17 [ 3930.650372] Lustre: DEBUG MARKER: == conf-sanity test 104a: Make sure user defined options are reflected in mount ========================================================== 23:58:35 (1763355515) [ 3944.418170] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 3944.420634] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 3944.422041] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 3944.424071] Lustre: Skipped 4 previous similar messages [ 3944.425708] Lustre: Skipped 4 previous similar messages [ 3948.478833] LustreError: 244187:0:(obd_class.h:479:obd_check_dev()) Device 10 not setup [ 3948.480611] LustreError: 244187:0:(obd_class.h:479:obd_check_dev()) Skipped 25 previous similar messages [ 3948.512663] Lustre: server umount lustre-MDT0000 complete [ 3949.539451] LustreError: 240855:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 3949.544636] LustreError: 240855:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 4 previous similar messages [ 3949.772387] LustreError: 242349:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1763355534 with bad export cookie 7946558116613125955 [ 3949.776032] LustreError: MGC192.168.201.140@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 3949.776917] LustreError: 242349:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 3964.384116] Lustre: lustre-MDT0001 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 3964.465564] Lustre: server umount lustre-MDT0001 complete [ 3980.256171] Lustre: lustre-OST0000 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 3980.294048] Lustre: server umount lustre-OST0000 complete [ 3987.744500] Lustre: server umount lustre-OST0001 complete [ 3989.817236] Lustre: DEBUG MARKER: oleg140-server.virtnet: executing set_hostid [ 3993.149197] Lustre: DEBUG MARKER: oleg140-server.virtnet: executing load_modules_local [ 3996.643452] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 3998.845909] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 4001.015389] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 4003.316670] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 4005.809930] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 4005.837809] LDISKFS-fs (dm-0): Mount option "noacl" will be removed by 3.5 [ 4005.837809] Contact linux-ldiskfs@vger.kernel.org if you think we should keep it. [ 4005.837809] [ 4005.846522] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: acl,user_xattr,errors=remount-ro,noacl,no_mbcache,nodelalloc [ 4005.931596] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 4005.945385] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 4005.981224] Lustre: lustre-MDT0000: new disk, initializing [ 4006.009347] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 4006.011366] Lustre: Skipped 3 previous similar messages [ 4006.018098] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 4007.381739] Lustre: DEBUG MARKER: oleg140-server.virtnet: executing set_default_debug -1 all [ 4011.570087] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 4011.594842] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 4011.634435] Lustre: 246973:0:(mgs_llog.c:1346:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 4011.663757] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [ 4011.665410] Lustre: Skipped 1 previous similar message [ 4011.705343] Lustre: lustre-MDT0001: new disk, initializing [ 4011.753900] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [ 4011.757275] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 4013.028253] Lustre: DEBUG MARKER: oleg140-server.virtnet: executing set_default_debug -1 all [ 4017.231412] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 4017.252239] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 4017.361663] Lustre: lustre-OST0000: new disk, initializing [ 4017.363436] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 4018.880084] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [ 4018.884356] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 4018.928684] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 4019.336485] Lustre: DEBUG MARKER: oleg140-server.virtnet: executing set_default_debug -1 all [ 4023.517442] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 4023.542697] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 4023.600122] Lustre: lustre-OST0001: new disk, initializing [ 4023.602277] Lustre: srv-lustre-OST0001: No data found on store. Initialize space. [ 4024.774714] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x00000002c0000400-0x0000000300000400]:1:ost [ 4024.778714] Lustre: cli-lustre-OST0001-super: Allocated super-sequence [0x00000002c0000400-0x0000000300000400]:1:ost] [ 4024.807532] Lustre: lustre-OST0001-osc-MDT0000: update sequence from 0x100010000 to 0x2c0000401 [ 4025.640115] Lustre: DEBUG MARKER: oleg140-server.virtnet: executing set_default_debug -1 all [ 4030.157862] Lustre: DEBUG MARKER: == conf-sanity test 104b: Mount uses last flock argument ========================================================== 00:00:14 (1763355614) [ 4034.099610] Lustre: DEBUG MARKER: == conf-sanity test 105: check file creation for ro and rw bind mnt pt ========================================================== 00:00:18 (1763355618) [ 4049.889332] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 4049.892341] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4049.896860] Lustre: Skipped 4 previous similar messages [ 4049.899633] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 4049.901658] Lustre: Skipped 8 previous similar messages [ 4051.757568] LustreError: 250398:0:(obd_class.h:479:obd_check_dev()) Device 19 not setup [ 4051.759847] LustreError: 250398:0:(obd_class.h:479:obd_check_dev()) Skipped 25 previous similar messages [ 4051.779855] Lustre: server umount lustre-OST0000 complete [ 4055.523287] LustreError: 248515:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-OST0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 4055.529224] LustreError: 248515:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 4 previous similar messages [ 4059.292959] Lustre: server umount lustre-MDT0000 complete [ 4060.609817] LustreError: 246965:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1763355645 with bad export cookie 7946558116613127509 [ 4060.611901] LustreError: MGC192.168.201.140@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 4060.614406] LustreError: 246965:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) Skipped 1 previous similar message [ 4066.797222] Lustre: server umount lustre-MDT0001 complete [ 4070.582796] Lustre: DEBUG MARKER: oleg140-server.virtnet: executing unload_modules_local [ 4080.343299] Lustre: server umount lustre-OST0001 complete [ 4082.454678] Lustre: DEBUG MARKER: oleg140-server.virtnet: executing set_hostid [ 4088.983566] Lustre: DEBUG MARKER: oleg140-server.virtnet: executing load_modules_local [ 4092.506369] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 4094.717826] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 4096.840923] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 4099.060313] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 4101.403167] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 4101.423553] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 4101.507442] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 4101.522217] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 4101.558846] Lustre: lustre-MDT0000: new disk, initializing [ 4101.592852] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 4102.923963] Lustre: DEBUG MARKER: oleg140-server.virtnet: executing set_default_debug -1 all [ 4107.021924] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 4107.044247] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 4107.084027] Lustre: 253870:0:(mgs_llog.c:1346:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 4107.210383] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 4108.573597] Lustre: DEBUG MARKER: oleg140-server.virtnet: executing set_default_debug -1 all [ 4111.829947] Lustre: DEBUG MARKER: oleg140-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 4113.139357] Lustre: DEBUG MARKER: oleg140-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 4115.455884] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 4115.477200] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 4116.670433] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 4116.705437] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 4117.556967] Lustre: DEBUG MARKER: oleg140-server.virtnet: executing set_default_debug -1 all [ 4120.737549] Lustre: DEBUG MARKER: oleg140-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 4127.201747] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 4127.201856] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4127.204875] LustreError: Skipped 2 previous similar messages [ 4127.207997] Lustre: Skipped 5 previous similar messages [ 4127.208575] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 4127.213646] Lustre: Skipped 8 previous similar messages [ 4129.179789] Lustre: server umount lustre-OST0000 complete [ 4136.604602] Lustre: server umount lustre-MDT0000 complete [ 4137.440997] LustreError: 253881:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 4137.447240] LustreError: 253881:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 3 previous similar messages [ 4137.885178] LustreError: 253861:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1763355723 with bad export cookie 7946558116613128979 [ 4137.888363] LustreError: MGC192.168.201.140@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 4137.889448] LustreError: 253861:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 4138.015031] Lustre: server umount lustre-MDT0001 complete [ 4141.674344] Lustre: DEBUG MARKER: oleg140-server.virtnet: executing unload_modules_local [ 4142.704319] Key type lgssc unregistered [ 4142.839346] LNet: 257249:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 4142.841430] LNetError: 257249:0:(acceptor.c:264:lnet_acceptor_remove_socket()) Interface ens2 not found [ 4142.849442] LNet: Removed LNI 192.168.201.140@tcp [ 4143.141124] Key type .llcrypt unregistered [ 4143.142190] Key type ._llcrypt unregistered [ 4153.622165] Key type ._llcrypt registered [ 4153.623195] Key type .llcrypt registered [ 4153.662821] Lustre: DEBUG MARKER: oleg140-server.virtnet: executing load_modules_local [ 4154.065662] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 4154.080832] alg: No test for adler32 (adler32-zlib) [ 4154.936711] Lustre: Lustre: Build Version: 2.16.61_41_g6a6a76f [ 4155.020203] LNet: Added LNI 192.168.201.140@tcp [8/256/0/180] [ 4156.600149] Key type lgssc registered [ 4156.949796] Lustre: Echo OBD driver; http://www.lustre.org/ [ 4159.458670] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 4159.465264] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 4160.606736] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 4161.838686] Lustre: DEBUG MARKER: oleg140-server.virtnet: executing set_default_debug -1 all [ 4164.568234] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 4164.677962] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 4165.925247] Lustre: DEBUG MARKER: oleg140-server.virtnet: executing set_default_debug -1 all [ 4167.734131] Lustre: DEBUG MARKER: oleg140-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 4168.975592] Lustre: DEBUG MARKER: oleg140-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 4171.220524] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 4171.338265] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 4173.096666] Lustre: DEBUG MARKER: oleg140-server.virtnet: executing set_default_debug -1 all [ 4174.830772] Lustre: DEBUG MARKER: oleg140-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 4175.152969] Lustre: Setting parameter lustre-OST0000.ost.unknown_param=50 in log lustre-OST0000 [ 4179.939066] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 4179.942312] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4179.947942] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 4181.868105] LustreError: 260962:0:(obd_class.h:479:obd_check_dev()) Device 21 not setup [ 4181.905215] Lustre: server umount lustre-OST0000 complete [ 4183.130250] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4183.130725] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 4183.134882] Lustre: Skipped 1 previous similar message [ 4183.138845] Lustre: Skipped 1 previous similar message [ 4188.641106] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 4188.643598] Lustre: Skipped 1 previous similar message [ 4189.247062] LustreError: 261171:0:(obd_class.h:479:obd_check_dev()) Device 9 not setup [ 4189.249158] LustreError: 261171:0:(obd_class.h:479:obd_check_dev()) Skipped 3 previous similar messages [ 4189.275157] Lustre: server umount lustre-MDT0000 complete [ 4190.494981] LustreError: 258649:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1763355775 with bad export cookie 17857894080766063496 [ 4190.498193] LustreError: 258649:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 4190.498333] LustreError: MGC192.168.201.140@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 4190.560133] LustreError: 261372:0:(obd_class.h:479:obd_check_dev()) Device 16 not setup [ 4190.562477] LustreError: 261372:0:(obd_class.h:479:obd_check_dev()) Skipped 5 previous similar messages [ 4190.610750] Lustre: server umount lustre-MDT0001 complete [ 4193.920204] Lustre: DEBUG MARKER: oleg140-server.virtnet: executing unload_modules_local [ 4194.901833] Key type lgssc unregistered [ 4195.020353] LNet: 261903:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 4195.022668] LNetError: 261903:0:(acceptor.c:264:lnet_acceptor_remove_socket()) Interface ens2 not found [ 4195.034302] LNet: Removed LNI 192.168.201.140@tcp [ 4195.293241] Key type .llcrypt unregistered [ 4195.294252] Key type ._llcrypt unregistered [ 4202.064767] Key type ._llcrypt registered [ 4202.065825] Key type .llcrypt registered [ 4202.104219] Lustre: DEBUG MARKER: oleg140-server.virtnet: executing load_modules_local [ 4202.408155] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 4202.416952] alg: No test for adler32 (adler32-zlib) [ 4203.261803] Lustre: Lustre: Build Version: 2.16.61_41_g6a6a76f [ 4203.332146] LNet: Added LNI 192.168.201.140@tcp [8/256/0/180] [ 4204.912176] Key type lgssc registered [ 4205.272273] Lustre: Echo OBD driver; http://www.lustre.org/ [ 4208.196296] Lustre: lustre-OST0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 4208.202598] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 4223.648411] LustreError: Failed to get MGS log lustre-sptlrpc and no local copy. [ 4228.768261] LustreError: 263053:0:(mgc_request.c:233:do_config_log_add()) MGC192.168.201.140@tcp: failed processing log, type 4: rc = -110 [ 4255.392226] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 4257.167053] Lustre: DEBUG MARKER: oleg140-server.virtnet: executing set_default_debug -1 all [ 4258.973611] Lustre: DEBUG MARKER: oleg140-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 4259.692336] LustreError: 263716:0:(obd_class.h:479:obd_check_dev()) Device 1 not setup [ 4259.733188] Lustre: server umount lustre-OST0000 complete [ 4265.211241] Lustre: DEBUG MARKER: oleg140-server.virtnet: executing unload_modules_local [ 4266.266140] Key type lgssc unregistered [ 4266.395405] LNet: 264552:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 4266.398075] LNetError: 264552:0:(acceptor.c:264:lnet_acceptor_remove_socket()) Interface ens2 not found [ 4266.407317] LNet: Removed LNI 192.168.201.140@tcp [ 4266.698747] Key type .llcrypt unregistered [ 4266.700095] Key type ._llcrypt unregistered [ 4280.298704] Key type ._llcrypt registered [ 4280.300195] Key type .llcrypt registered [ 4280.339963] Lustre: DEBUG MARKER: oleg140-server.virtnet: executing load_modules_local [ 4280.831106] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 4280.850085] alg: No test for adler32 (adler32-zlib) [ 4281.714933] Lustre: Lustre: Build Version: 2.16.61_41_g6a6a76f [ 4281.816245] LNet: Added LNI 192.168.201.140@tcp [8/256/0/180] [ 4283.392194] Key type lgssc registered [ 4283.746796] Lustre: Echo OBD driver; http://www.lustre.org/ [ 4288.031239] loop0: detected capacity change from 0 to 204800000 [ 4288.132503] blk_update_request: operation not supported error, dev loop0, sector 399872 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [ 4288.136785] blk_update_request: operation not supported error, dev loop0, sector 208 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [ 4288.256655] blk_update_request: operation not supported error, dev loop0, sector 160232 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [ 4288.276274] LDISKFS-fs (loop0): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 4288.615561] loop0: detected capacity change from 0 to 204800000 [ 4288.677993] blk_update_request: operation not supported error, dev loop0, sector 399872 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [ 4288.681705] blk_update_request: operation not supported error, dev loop0, sector 208 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [ 4288.812903] blk_update_request: operation not supported error, dev loop0, sector 160232 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [ 4288.836549] LDISKFS-fs (loop0): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 4289.183601] loop0: detected capacity change from 0 to 204800000 [ 4289.246868] blk_update_request: operation not supported error, dev loop0, sector 399872 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [ 4289.250775] blk_update_request: operation not supported error, dev loop0, sector 8224 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [ 4289.300168] blk_update_request: operation not supported error, dev loop0, sector 58264 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [ 4289.345393] LDISKFS-fs (loop0): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 4289.705725] loop0: detected capacity change from 0 to 204800000 [ 4289.759960] blk_update_request: operation not supported error, dev loop0, sector 399872 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [ 4289.845374] LDISKFS-fs (loop0): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 4290.173401] loop0: detected capacity change from 0 to 204800000 [ 4290.188326] LDISKFS-fs (loop0): mounted filesystem with ordered data mode. Opts: (null) [ 4291.551148] loop0: detected capacity change from 0 to 204800000 [ 4291.561508] LDISKFS-fs (loop0): mounted filesystem with ordered data mode. Opts: (null) [ 4292.854761] loop0: detected capacity change from 0 to 204800000 [ 4292.867075] LDISKFS-fs (loop0): mounted filesystem with ordered data mode. Opts: (null) [ 4294.203898] loop0: detected capacity change from 0 to 204800000 [ 4294.213227] LDISKFS-fs (loop0): mounted filesystem with ordered data mode. Opts: (null) [ 4295.477848] loop0: detected capacity change from 0 to 204800000 [ 4295.512984] LDISKFS-fs (loop0): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 4295.538472] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 4295.546752] LDISKFS-fs (loop0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 4295.575814] Lustre: 267926:0:(osd_scrub.c:1842:osd_index_restore()) lustre-MDT0000: restore index 'fld' with [0x200000001:0x3:0x0]: rc = 0 [ 4295.580964] Lustre: 267926:0:(osd_scrub.c:1842:osd_index_restore()) lustre-MDT0000: restore index 'nodemap' with [0x200000003:0x3:0x0]: rc = 0 [ 4295.586384] Lustre: 267926:0:(osd_scrub.c:1842:osd_index_restore()) lustre-MDT0000: restore index '0x10000-MDT0000' with [0x200000005:0x1:0x0]: rc = 0 [ 4295.591850] Lustre: 267926:0:(osd_scrub.c:1842:osd_index_restore()) lustre-MDT0000: restore index '0x2010000-MDT0000' with [0x200000005:0x3:0x0]: rc = 0 [ 4295.595565] Lustre: 267926:0:(osd_scrub.c:1842:osd_index_restore()) lustre-MDT0000: restore index '0x1010000' with [0x200000003:0xb:0x0]: rc = 0 [ 4295.599481] Lustre: 267926:0:(osd_scrub.c:1842:osd_index_restore()) lustre-MDT0000: restore index '0x1010000-MDT0000' with [0x200000005:0x2:0x0]: rc = 0 [ 4295.604235] Lustre: 267926:0:(osd_scrub.c:1842:osd_index_restore()) lustre-MDT0000: restore index '0x10000' with [0x200000003:0xa:0x0]: rc = 0 [ 4295.607203] Lustre: 267926:0:(osd_scrub.c:1842:osd_index_restore()) lustre-MDT0000: restore index '0x2010000' with [0x200000003:0xc:0x0]: rc = 0 [ 4295.610468] Lustre: 267926:0:(osd_scrub.c:1842:osd_index_restore()) lustre-MDT0000: restore index '0x20000' with [0x200000006:0x20000:0x0]: rc = 0 [ 4295.613870] Lustre: 267926:0:(osd_scrub.c:1842:osd_index_restore()) lustre-MDT0000: restore index '0x2020000' with [0x200000006:0x2020000:0x0]: rc = 0 [ 4295.619040] Lustre: 267926:0:(osd_scrub.c:1842:osd_index_restore()) lustre-MDT0000: restore index '0x1020000' with [0x200000006:0x1020000:0x0]: rc = 0 [ 4295.623350] Lustre: 267926:0:(osd_scrub.c:1842:osd_index_restore()) lustre-MDT0000: restore index '0x1010000' with [0x200000006:0x1010000:0x0]: rc = 0 [ 4295.626693] Lustre: 267926:0:(osd_scrub.c:1842:osd_index_restore()) lustre-MDT0000: restore index '0x10000' with [0x200000006:0x10000:0x0]: rc = 0 [ 4295.629852] Lustre: 267926:0:(osd_scrub.c:1842:osd_index_restore()) lustre-MDT0000: restore index '0x2010000' with [0x200000006:0x2010000:0x0]: rc = 0 [ 4298.165102] LustreError: 268192:0:(obd_class.h:479:obd_check_dev()) Device 2 not setup [ 4298.208000] Lustre: server umount MGS complete [ 4298.513380] loop0: detected capacity change from 0 to 204800000 [ 4298.554703] LDISKFS-fs (loop0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 4298.721414] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 4298.739984] Lustre: lustre-MDT0000: Aborting client recovery [ 4299.063250] loop1: detected capacity change from 0 to 204800000 [ 4299.126189] LDISKFS-fs (loop1): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 4299.146668] LDISKFS-fs (loop1): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 4299.166276] Lustre: 268361:0:(osd_scrub.c:1842:osd_index_restore()) lustre-MDT0001: restore index 'fld' with [0x200000001:0x3:0x0]: rc = 0 [ 4299.171165] Lustre: 268361:0:(osd_scrub.c:1842:osd_index_restore()) lustre-MDT0001: restore index '0x2010000-MDT0001' with [0x200000005:0x3:0x0]: rc = 0 [ 4299.174610] Lustre: 268361:0:(osd_scrub.c:1842:osd_index_restore()) lustre-MDT0001: restore index '0x1010000-MDT0001' with [0x200000005:0x2:0x0]: rc = 0 [ 4299.178134] Lustre: 268361:0:(osd_scrub.c:1842:osd_index_restore()) lustre-MDT0001: restore index '0x2010000' with [0x200000003:0x6:0x0]: rc = 0 [ 4299.181474] Lustre: 268361:0:(osd_scrub.c:1842:osd_index_restore()) lustre-MDT0001: restore index '0x10000-MDT0001' with [0x200000005:0x1:0x0]: rc = 0 [ 4299.185307] Lustre: 268361:0:(osd_scrub.c:1842:osd_index_restore()) lustre-MDT0001: restore index '0x1010000' with [0x200000003:0x5:0x0]: rc = 0 [ 4299.188935] Lustre: 268361:0:(osd_scrub.c:1842:osd_index_restore()) lustre-MDT0001: restore index '0x10000' with [0x200000003:0x4:0x0]: rc = 0 [ 4299.195943] Lustre: Found index 1 for lustre-MDT0001, updating log [ 4299.267612] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 4299.289676] Lustre: lustre-MDT0001: Aborting client recovery [ 4299.618672] loop2: detected capacity change from 0 to 204800000 [ 4299.682631] LDISKFS-fs (loop2): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 4299.723460] LDISKFS-fs (loop2): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 4299.742237] Lustre: 268445:0:(osd_scrub.c:1842:osd_index_restore()) lustre-OST0000: restore index 'fld' with [0x200000001:0x3:0x0]: rc = 0 [ 4299.745877] Lustre: 268445:0:(osd_scrub.c:1842:osd_index_restore()) lustre-OST0000: restore index '0x20000' with [0x200000003:0x5:0x0]: rc = 0 [ 4299.749967] Lustre: 268445:0:(osd_scrub.c:1842:osd_index_restore()) lustre-OST0000: restore index '0x1020000-OST0000' with [0x200000005:0x2:0x0]: rc = 0 [ 4299.753668] Lustre: 268445:0:(osd_scrub.c:1842:osd_index_restore()) lustre-OST0000: restore index '0x20000-OST0000' with [0x200000005:0x1:0x0]: rc = 0 [ 4299.758383] Lustre: 268445:0:(osd_scrub.c:1842:osd_index_restore()) lustre-OST0000: restore index '0x2020000' with [0x200000003:0x7:0x0]: rc = 0 [ 4299.761821] Lustre: 268445:0:(osd_scrub.c:1842:osd_index_restore()) lustre-OST0000: restore index '0x1020000' with [0x200000003:0x6:0x0]: rc = 0 [ 4299.766748] Lustre: 268445:0:(osd_scrub.c:1842:osd_index_restore()) lustre-OST0000: restore index '0x2020000-OST0000' with [0x200000005:0x3:0x0]: rc = 0 [ 4299.836495] Lustre: Found index 0 for lustre-OST0000, updating log [ 4299.876911] Lustre: lustre-OST0000: Aborting recovery [ 4300.194451] loop3: detected capacity change from 0 to 204800000 [ 4300.246874] LDISKFS-fs (loop3): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 4300.268368] LDISKFS-fs (loop3): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 4300.290064] Lustre: 268532:0:(osd_scrub.c:1842:osd_index_restore()) lustre-OST0001: restore index 'fld' with [0x200000001:0x3:0x0]: rc = 0 [ 4300.295335] Lustre: 268532:0:(osd_scrub.c:1842:osd_index_restore()) lustre-OST0001: restore index '0x20000-OST0001' with [0x200000005:0x1:0x0]: rc = 0 [ 4300.301204] Lustre: 268532:0:(osd_scrub.c:1842:osd_index_restore()) lustre-OST0001: restore index '0x2020000-OST0001' with [0x200000005:0x3:0x0]: rc = 0 [ 4300.307111] Lustre: 268532:0:(osd_scrub.c:1842:osd_index_restore()) lustre-OST0001: restore index '0x2020000' with [0x200000003:0x7:0x0]: rc = 0 [ 4300.310795] Lustre: 268532:0:(osd_scrub.c:1842:osd_index_restore()) lustre-OST0001: restore index '0x1020000-OST0001' with [0x200000005:0x2:0x0]: rc = 0 [ 4300.314547] Lustre: 268532:0:(osd_scrub.c:1842:osd_index_restore()) lustre-OST0001: restore index '0x20000' with [0x200000003:0x5:0x0]: rc = 0 [ 4300.318037] Lustre: 268532:0:(osd_scrub.c:1842:osd_index_restore()) lustre-OST0001: restore index '0x1020000' with [0x200000003:0x6:0x0]: rc = 0 [ 4300.354503] Lustre: lustre-OST0001: Imperative Recovery not enabled, recovery window 60-180 [ 4300.357472] Lustre: Skipped 1 previous similar message [ 4305.384304] LustreError: 268304:0:(lod_dev.c:510:lod_sub_recovery_thread()) lustre-MDT0001-osp-MDT0000: get update log duration 7, retries 0, failed: rc = -108 [ 4308.450668] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x0:6 to 0x0:33) [ 4308.451313] Lustre: lustre-OST0001: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x0:6 to 0x0:33) [ 4308.460418] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000bd0 [ 4308.468695] Lustre: lustre-OST0001-osc-MDT0000: update sequence from 0x100010000 to 0x2c0000bd0 [ 4309.475294] Lustre: lustre-OST0001: new connection from lustre-MDT0001-mdtlov (cleaning up unused objects from 0x2c0000400:5 to 0x2c0000400:33) [ 4309.475362] Lustre: lustre-OST0000: new connection from lustre-MDT0001-mdtlov (cleaning up unused objects from 0x280000400:5 to 0x280000400:33) [ 4310.958034] LustreError: 268272:0:(mdt_identity.c:115:mdt_identity_do_upcall()) lustre-MDT0000: error invoking upcall /root/Work/Lustre/R5/lustre-release/lustre/tests/../utils/l_getidentity lustre-MDT0000 0: rc -2; check /proc/fs/lustre/mdt/lustre-MDT0000/identity_upcall, time 302us: rc = -2 [ 4324.833356] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 4324.833411] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4324.834073] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 4324.841444] Lustre: Skipped 2 previous similar messages [ 4328.253482] LustreError: 268890:0:(obd_class.h:479:obd_check_dev()) Device 10 not setup [ 4328.256040] LustreError: 268890:0:(obd_class.h:479:obd_check_dev()) Skipped 1 previous similar message [ 4328.289795] Lustre: server umount lustre-MDT0000 complete [ 4328.609702] LustreError: 268840:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1763355913 with bad export cookie 11413366980090511644 [ 4328.611154] LustreError: MGC192.168.201.140@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 4328.614483] LustreError: 268840:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) Skipped 2 previous similar messages [ 4329.952813] LustreError: 268769:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 4329.953273] Lustre: lustre-MDT0001-lwp-OST0000: Connection to lustre-MDT0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4329.953626] Lustre: lustre-MDT0001: Not available for connect from 0@lo (stopping) [ 4329.953635] Lustre: Skipped 3 previous similar messages [ 4329.958287] LustreError: 268769:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 3 previous similar messages [ 4329.962834] Lustre: Skipped 2 previous similar messages [ 4333.984652] LustreError: 268769:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 4333.984702] Lustre: lustre-MDT0001: Not available for connect from 0@lo (stopping) [ 4333.988557] LustreError: 268769:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 1 previous similar message [ 4333.991944] Lustre: Skipped 2 previous similar messages [ 4334.907432] LustreError: 268943:0:(obd_class.h:479:obd_check_dev()) Device 18 not setup [ 4334.909981] LustreError: 268943:0:(obd_class.h:479:obd_check_dev()) Skipped 7 previous similar messages [ 4334.966376] Lustre: server umount lustre-MDT0001 complete [ 4341.356040] LustreError: 268994:0:(obd_class.h:479:obd_check_dev()) Device 23 not setup [ 4341.357904] LustreError: 268994:0:(obd_class.h:479:obd_check_dev()) Skipped 7 previous similar messages [ 4341.384936] Lustre: server umount lustre-OST0000 complete [ 4356.064111] Lustre: lustre-OST0001 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 4356.082527] LustreError: 269045:0:(obd_class.h:479:obd_check_dev()) Device 2 not setup [ 4356.084249] LustreError: 269045:0:(obd_class.h:479:obd_check_dev()) Skipped 3 previous similar messages [ 4356.125313] Lustre: server umount lustre-OST0001 complete [ 4358.677297] Lustre: DEBUG MARKER: == conf-sanity test 109a: test lctl clear_conf fsname ==== 00:05:43 (1763355943) [ 4368.094996] Lustre: DEBUG MARKER: oleg140-server.virtnet: executing set_hostid [ 4371.452698] Lustre: DEBUG MARKER: oleg140-server.virtnet: executing load_modules_local [ 4375.150846] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 4377.277251] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 4379.306423] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 4381.335903] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 4383.463190] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 4383.486581] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 4383.559216] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 4383.572225] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 4383.604331] Lustre: lustre-MDT0000: new disk, initializing [ 4383.627403] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 4383.635912] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 4384.835511] Lustre: DEBUG MARKER: oleg140-server.virtnet: executing set_default_debug -1 all [ 4388.791101] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 4388.811506] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 4388.840560] Lustre: 272592:0:(mgs_llog.c:1346:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 4388.859768] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [ 4388.861624] Lustre: Skipped 1 previous similar message [ 4388.896765] Lustre: lustre-MDT0001: new disk, initializing [ 4388.926609] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 4388.937979] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [ 4388.942030] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 4390.113541] Lustre: DEBUG MARKER: oleg140-server.virtnet: executing set_default_debug -1 all [ 4393.203407] Lustre: DEBUG MARKER: oleg140-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 4394.529383] Lustre: DEBUG MARKER: oleg140-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 4396.812540] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 4396.832972] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 4396.917988] Lustre: lustre-OST0000: new disk, initializing [ 4396.920712] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 4396.948332] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 4398.197693] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [ 4398.202122] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 4398.234512] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 4398.766644] Lustre: DEBUG MARKER: oleg140-server.virtnet: executing set_default_debug -1 all [ 4401.900404] Lustre: DEBUG MARKER: oleg140-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 4404.234284] Lustre: Setting parameter lustre-MDT0000.mdd.atime_diff=62 in log lustre-MDT0000 [ 4414.374056] Lustre: Modifying parameter lustre-MDT0000.mdd.atime_diff=63 in log lustre-MDT0000 [ 4421.404197] Lustre: Setting parameter lustre-client.llite.max_read_ahead_mb=32 in log lustre-client [ 4429.007318] Lustre: Modifying parameter lustre-client.llite.max_read_ahead_mb=64 in log lustre-client [ 4468.197062] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4468.198536] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 4468.202868] Lustre: Skipped 1 previous similar message [ 4472.174211] LustreError: 276398:0:(obd_class.h:479:obd_check_dev()) Device 19 not setup [ 4472.176145] LustreError: 276398:0:(obd_class.h:479:obd_check_dev()) Skipped 5 previous similar messages [ 4472.213267] Lustre: server umount lustre-OST0000 complete [ 4475.872897] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 4475.875392] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4475.879766] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 4475.881425] Lustre: Skipped 1 previous similar message [ 4479.644053] Lustre: server umount lustre-MDT0000 complete [ 4480.996523] LustreError: 273373:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1763356066 with bad export cookie 11413366980090514346 [ 4480.999772] LustreError: 273373:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) Skipped 2 previous similar messages [ 4481.000122] LustreError: MGC192.168.201.140@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 4481.132658] Lustre: server umount lustre-MDT0001 complete [ 4484.216317] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 4485.531563] Lustre: DEBUG MARKER: oleg140-server.virtnet: executing set_default_debug -1 all [ 4487.200608] Lustre: server umount MGS complete [ 4491.190026] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 4491.353837] LustreError: 278362:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 4491.361215] LustreError: 278374:0:(qmt_pool.c:1523:qmt_pool_add_rem()) lustre-QMT0000: can't remove lustre-OST0000_UUID pool 'pool1': rc = -22 [ 4491.385719] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 4492.762544] Lustre: DEBUG MARKER: oleg140-server.virtnet: executing set_default_debug -1 all [ 4495.770767] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 4497.266529] Lustre: DEBUG MARKER: oleg140-server.virtnet: executing set_default_debug -1 all [ 4499.097292] Lustre: DEBUG MARKER: oleg140-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 4500.283357] Lustre: DEBUG MARKER: oleg140-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 4502.627666] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 4504.866050] Lustre: DEBUG MARKER: oleg140-server.virtnet: executing set_default_debug -1 all [ 4506.786484] Lustre: DEBUG MARKER: oleg140-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 4537.827553] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4537.828481] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 4537.829057] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 4537.829061] Lustre: Skipped 2 previous similar messages [ 4537.833544] Lustre: Skipped 2 previous similar messages [ 4539.246114] LustreError: 281042:0:(obd_class.h:479:obd_check_dev()) Device 21 not setup [ 4539.248995] LustreError: 281042:0:(obd_class.h:479:obd_check_dev()) Skipped 19 previous similar messages [ 4539.293154] Lustre: server umount lustre-OST0000 complete [ 4541.920902] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 4541.923716] LustreError: Skipped 1 previous similar message [ 4546.710955] Lustre: server umount lustre-MDT0000 complete [ 4547.936417] LustreError: 278343:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1763356133 with bad export cookie 11413366980090516187 [ 4547.939867] LustreError: 278343:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) Skipped 3 previous similar messages [ 4547.940909] LustreError: MGC192.168.201.140@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 4548.073126] Lustre: server umount lustre-MDT0001 complete [ 4551.759986] Lustre: DEBUG MARKER: oleg140-server.virtnet: executing unload_modules_local [ 4552.869583] Key type lgssc unregistered [ 4553.020438] LNet: 281977:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 4553.022693] LNetError: 281977:0:(acceptor.c:264:lnet_acceptor_remove_socket()) Interface ens2 not found [ 4553.031440] LNet: Removed LNI 192.168.201.140@tcp [ 4553.375710] Key type .llcrypt unregistered [ 4553.377487] Key type ._llcrypt unregistered [ 4565.937429] Key type ._llcrypt registered [ 4565.938562] Key type .llcrypt registered [ 4565.977405] Lustre: DEBUG MARKER: oleg140-server.virtnet: executing set_hostid [ 4572.590234] Lustre: DEBUG MARKER: oleg140-server.virtnet: executing load_modules_local [ 4572.913899] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 4572.935932] alg: No test for adler32 (adler32-zlib) [ 4573.809360] Lustre: Lustre: Build Version: 2.16.61_41_g6a6a76f [ 4573.910634] LNet: Added LNI 192.168.201.140@tcp [8/256/0/180] [ 4575.496176] Key type lgssc registered [ 4575.867145] Lustre: Echo OBD driver; http://www.lustre.org/ [ 4578.978632] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 4581.270074] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 4583.481340] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 4585.721616] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 4588.176769] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 4588.198699] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 4588.204068] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 4589.296831] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 4589.311169] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 4589.349386] Lustre: lustre-MDT0000: new disk, initializing [ 4589.377525] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 4589.384374] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 4590.765664] Lustre: DEBUG MARKER: oleg140-server.virtnet: executing set_default_debug -1 all [ 4595.141520] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 4595.167309] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 4595.204539] Lustre: 285880:0:(mgs_llog.c:1346:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 4595.230556] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [ 4595.233615] Lustre: Skipped 1 previous similar message [ 4595.277888] Lustre: lustre-MDT0001: new disk, initializing [ 4595.316963] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 4595.331696] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [ 4595.335692] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 4596.775597] Lustre: DEBUG MARKER: oleg140-server.virtnet: executing set_default_debug -1 all [ 4600.022199] Lustre: DEBUG MARKER: oleg140-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 4601.327046] Lustre: DEBUG MARKER: oleg140-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 4603.723778] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 4603.746625] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 4603.856064] Lustre: lustre-OST0000: new disk, initializing [ 4603.858389] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 4603.887229] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 4605.858555] Lustre: DEBUG MARKER: oleg140-server.virtnet: executing set_default_debug -1 all [ 4607.878876] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [ 4607.884352] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 4607.933416] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 4609.157900] Lustre: DEBUG MARKER: oleg140-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 4612.710736] Lustre: Setting parameter lustre-MDT0000.mdd.atime_diff=62 in log lustre-MDT0000 [ 4617.878469] Lustre: Modifying parameter lustre-MDT0000.mdd.atime_diff=63 in log lustre-MDT0000 [ 4626.313057] Lustre: Setting parameter lustre-client.llite.max_read_ahead_mb=32 in log lustre-client [ 4669.410876] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 4669.412339] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4669.415108] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 4669.417711] Lustre: Skipped 1 previous similar message [ 4669.421312] Lustre: Skipped 1 previous similar message [ 4673.711091] LustreError: 289539:0:(obd_class.h:479:obd_check_dev()) Device 19 not setup [ 4673.752064] Lustre: server umount lustre-OST0000 complete [ 4677.600984] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 4677.603560] LustreError: Skipped 1 previous similar message [ 4677.605180] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4677.609293] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 4679.649638] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 4679.649963] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4681.279204] LustreError: 289740:0:(obd_class.h:479:obd_check_dev()) Device 22 not setup [ 4681.281177] LustreError: 289740:0:(obd_class.h:479:obd_check_dev()) Skipped 3 previous similar messages [ 4681.314246] Lustre: server umount lustre-MDT0000 complete [ 4682.689292] LustreError: 286660:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1763356267 with bad export cookie 17519759127865570510 [ 4682.691984] LustreError: MGC192.168.201.140@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 4682.694160] LustreError: 286660:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) Skipped 2 previous similar messages [ 4682.776784] LustreError: 289941:0:(obd_class.h:479:obd_check_dev()) Device 21 not setup [ 4682.779629] LustreError: 289941:0:(obd_class.h:479:obd_check_dev()) Skipped 5 previous similar messages [ 4682.842091] Lustre: server umount lustre-MDT0001 complete [ 4686.045318] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 4687.533297] Lustre: DEBUG MARKER: oleg140-server.virtnet: executing set_default_debug -1 all [ 4689.222841] LustreError: 290881:0:(obd_class.h:479:obd_check_dev()) Device 2 not setup [ 4689.225921] LustreError: 290881:0:(obd_class.h:479:obd_check_dev()) Skipped 7 previous similar messages [ 4689.251147] Lustre: server umount MGS complete [ 4693.343764] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 4693.510785] LustreError: 291516:0:(qmt_pool.c:1523:qmt_pool_add_rem()) lustre-QMT0000: can't remove lustre-OST0000_UUID pool 'pool1': rc = -22 [ 4693.535531] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 4694.881731] Lustre: DEBUG MARKER: oleg140-server.virtnet: executing set_default_debug -1 all [ 4697.849031] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 4697.981128] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 4699.313615] Lustre: DEBUG MARKER: oleg140-server.virtnet: executing set_default_debug -1 all [ 4701.251587] Lustre: DEBUG MARKER: oleg140-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 4702.582498] Lustre: DEBUG MARKER: oleg140-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 4705.033720] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 4707.111822] Lustre: DEBUG MARKER: oleg140-server.virtnet: executing set_default_debug -1 all [ 4709.064938] Lustre: DEBUG MARKER: oleg140-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 4739.043203] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 4739.043305] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4739.052547] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 4739.054162] Lustre: Skipped 2 previous similar messages [ 4744.163581] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 4744.167557] Lustre: Skipped 1 previous similar message [ 4744.495129] LustreError: 294184:0:(obd_class.h:479:obd_check_dev()) Device 21 not setup [ 4744.498327] LustreError: 294184:0:(obd_class.h:479:obd_check_dev()) Skipped 1 previous similar message [ 4744.541692] Lustre: server umount lustre-OST0000 complete [ 4749.282344] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 4749.282772] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4749.284575] LustreError: Skipped 1 previous similar message [ 4749.289678] Lustre: Skipped 1 previous similar message [ 4752.103677] Lustre: server umount lustre-MDT0000 complete [ 4753.390130] LustreError: 291484:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1763356338 with bad export cookie 17519759127865572337 [ 4753.392624] LustreError: MGC192.168.201.140@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 4753.394364] LustreError: 291484:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) Skipped 3 previous similar messages [ 4753.470033] LustreError: 294586:0:(obd_class.h:479:obd_check_dev()) Device 16 not setup [ 4753.471777] LustreError: 294586:0:(obd_class.h:479:obd_check_dev()) Skipped 9 previous similar messages [ 4753.533773] Lustre: server umount lustre-MDT0001 complete [ 4757.528709] Lustre: DEBUG MARKER: oleg140-server.virtnet: executing unload_modules_local [ 4758.709924] Key type lgssc unregistered [ 4758.846478] LNet: 295116:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 4758.848996] LNetError: 295116:0:(acceptor.c:264:lnet_acceptor_remove_socket()) Interface ens2 not found [ 4758.856498] LNet: Removed LNI 192.168.201.140@tcp [ 4759.190764] Key type .llcrypt unregistered [ 4759.191898] Key type ._llcrypt unregistered [ 4766.510943] Key type ._llcrypt registered [ 4766.512052] Key type .llcrypt registered [ 4766.559781] Lustre: DEBUG MARKER: oleg140-server.virtnet: executing set_hostid [ 4772.569725] Lustre: DEBUG MARKER: oleg140-server.virtnet: executing load_modules_local [ 4772.996795] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 4773.030329] alg: No test for adler32 (adler32-zlib) [ 4773.890547] Lustre: Lustre: Build Version: 2.16.61_41_g6a6a76f [ 4773.965845] LNet: Added LNI 192.168.201.140@tcp [8/256/0/180] [ 4775.544189] Key type lgssc registered [ 4775.884778] Lustre: Echo OBD driver; http://www.lustre.org/ [ 4778.764932] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 4780.750746] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 4783.758136] Lustre: DEBUG MARKER: oleg140-server.virtnet: executing load_modules_local [ 4786.267817] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 4788.506640] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 4788.534345] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 4788.539479] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 4789.626747] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 4789.640400] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 4789.685951] Lustre: lustre-MDT0000: new disk, initializing [ 4789.711919] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 4789.720637] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 4791.045907] Lustre: DEBUG MARKER: oleg140-server.virtnet: executing set_default_debug -1 all [ 4795.135241] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 4795.162194] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 4795.259194] Lustre: lustre-OST0000: new disk, initializing [ 4795.262279] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 4795.265317] Lustre: Skipped 1 previous similar message [ 4795.288628] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 4797.205490] Lustre: DEBUG MARKER: oleg140-server.virtnet: executing set_default_debug -1 all [ 4798.079277] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:0:ost [ 4798.082505] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:0:ost] [ 4798.092659] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x240000400 [ 4982.753799] Lustre: lustre-MDT0000-lwp-OST0000: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4982.759266] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 4986.565982] LustreError: 300031:0:(obd_class.h:479:obd_check_dev()) Device 13 not setup [ 4986.697371] Lustre: server umount lustre-MDT0000 complete [ 4987.947652] LustreError: 298280:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1763356573 with bad export cookie 13698912152068802909 [ 4987.949565] LustreError: MGC192.168.201.140@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 4987.953471] LustreError: 298280:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) Skipped 2 previous similar messages [ 4987.979302] LustreError: 300233:0:(obd_class.h:479:obd_check_dev()) Device 2 not setup [ 4987.980680] LustreError: 300233:0:(obd_class.h:479:obd_check_dev()) Skipped 3 previous similar messages [ 4988.023916] Lustre: server umount lustre-OST0000 complete [ 4996.158093] Lustre: DEBUG MARKER: oleg140-server.virtnet: executing set_hostid [ 4999.225722] Lustre: DEBUG MARKER: oleg140-server.virtnet: executing load_modules_local [ 5002.855551] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 5004.883645] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 5006.928832] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 5009.189034] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 5011.359943] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 5011.381221] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 5011.469114] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 5011.482607] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 5011.519607] Lustre: lustre-MDT0000: new disk, initializing [ 5011.545035] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 5011.550760] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 5012.947920] Lustre: DEBUG MARKER: oleg140-server.virtnet: executing set_default_debug -1 all [ 5017.255732] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 5017.286363] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 5017.325204] Lustre: 303269:0:(mgs_llog.c:1346:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 5017.347467] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [ 5017.349306] Lustre: Skipped 1 previous similar message [ 5017.389424] Lustre: lustre-MDT0001: new disk, initializing [ 5017.424327] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 5017.436168] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [ 5017.440565] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 5018.803716] Lustre: DEBUG MARKER: oleg140-server.virtnet: executing set_default_debug -1 all [ 5021.899592] Lustre: DEBUG MARKER: oleg140-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 5023.202694] Lustre: DEBUG MARKER: oleg140-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 5025.663556] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 5025.685141] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 5025.794360] Lustre: lustre-OST0000: new disk, initializing [ 5025.797216] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 5025.823859] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 5027.261098] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [ 5027.265050] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 5027.305985] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 5027.921502] Lustre: DEBUG MARKER: oleg140-server.virtnet: executing set_default_debug -1 all [ 5031.263077] Lustre: DEBUG MARKER: oleg140-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 5032.656489] Lustre: DEBUG MARKER: oleg140-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 [ 5032.734768] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [ 5034.115619] Lustre: DEBUG MARKER: oleg140-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid 50 [ 5034.193219] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [ 5037.540399] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 5037.541395] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5037.543525] LustreError: Skipped 1 previous similar message [ 5037.547665] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 5037.551218] Lustre: Skipped 1 previous similar message [ 5042.661106] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 5042.662923] Lustre: Skipped 1 previous similar message [ 5047.779057] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 5047.781452] Lustre: Skipped 1 previous similar message [ 5049.312166] Lustre: lustre-OST0000 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 5049.331608] LustreError: 306208:0:(obd_class.h:479:obd_check_dev()) Device 19 not setup [ 5049.333577] LustreError: 306208:0:(obd_class.h:479:obd_check_dev()) Skipped 3 previous similar messages [ 5049.377576] Lustre: server umount lustre-OST0000 complete [ 5052.897820] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5052.898591] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 5052.901444] Lustre: Skipped 2 previous similar messages [ 5052.906835] Lustre: Skipped 1 previous similar message [ 5056.771892] LustreError: 306410:0:(obd_class.h:479:obd_check_dev()) Device 22 not setup [ 5056.774524] LustreError: 306410:0:(obd_class.h:479:obd_check_dev()) Skipped 3 previous similar messages [ 5056.806412] Lustre: server umount lustre-MDT0000 complete [ 5058.152163] LustreError: 303259:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1763356643 with bad export cookie 13698912152071743679 [ 5058.155791] LustreError: MGC192.168.201.140@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 5058.156931] LustreError: 303259:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) Skipped 3 previous similar messages [ 5058.298745] Lustre: server umount lustre-MDT0001 complete [ 5059.528948] Lustre: DEBUG MARKER: SKIP: conf-sanity test_111 skipping SLOW test 111 [ 5060.112784] Lustre: DEBUG MARKER: == conf-sanity test 112a: mount OST with no_create option ========================================================== 00:17:24 (1763356644) [ 5062.514389] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 5062.706562] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 5064.164713] Lustre: DEBUG MARKER: oleg140-server.virtnet: executing set_default_debug -1 all [ 5067.261618] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 5068.873043] Lustre: DEBUG MARKER: oleg140-server.virtnet: executing set_default_debug -1 all [ 5070.772756] Lustre: DEBUG MARKER: oleg140-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 5072.080644] Lustre: DEBUG MARKER: oleg140-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 5074.528386] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 5076.745676] Lustre: DEBUG MARKER: oleg140-server.virtnet: executing set_default_debug -1 all [ 5078.668206] Lustre: DEBUG MARKER: oleg140-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 5081.060101] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 5081.082351] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 5081.142345] Lustre: lustre-OST0001: new disk, initializing [ 5081.145271] Lustre: srv-lustre-OST0001: No data found on store. Initialize space. [ 5081.176138] Lustre: lustre-OST0001: Imperative Recovery not enabled, recovery window 60-180 [ 5081.179353] Lustre: Skipped 2 previous similar messages [ 5083.209202] Lustre: DEBUG MARKER: oleg140-server.virtnet: executing set_default_debug -1 all [ 5088.260887] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x00000002c0000400-0x0000000300000400]:1:ost [ 5088.263893] Lustre: cli-lustre-OST0001-super: Allocated super-sequence [0x00000002c0000400-0x0000000300000400]:1:ost] [ 5088.281389] LustreError: 310241:0:(osp_precreate.c:969:osp_precreate_cleanup_orphans()) lustre-OST0001-osc-MDT0001: cannot cleanup orphans: rc = -1 [ 5089.315651] LustreError: 310241:0:(osp_precreate.c:969:osp_precreate_cleanup_orphans()) lustre-OST0001-osc-MDT0001: cannot cleanup orphans: rc = -1 [ 5089.320848] LustreError: 310241:0:(osp_precreate.c:969:osp_precreate_cleanup_orphans()) Skipped 2 previous similar messages [ 5090.339828] LustreError: 310245:0:(osp_precreate.c:969:osp_precreate_cleanup_orphans()) lustre-OST0001-osc-MDT0000: cannot cleanup orphans: rc = -1 [ 5090.342526] LustreError: 310245:0:(osp_precreate.c:969:osp_precreate_cleanup_orphans()) Skipped 1 previous similar message [ 5092.278502] Lustre: DEBUG MARKER: oleg140-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 [ 5092.363909] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [ 5092.390102] LustreError: 310245:0:(osp_precreate.c:969:osp_precreate_cleanup_orphans()) lustre-OST0001-osc-MDT0000: cannot cleanup orphans: rc = -1 [ 5092.393883] LustreError: 310245:0:(osp_precreate.c:969:osp_precreate_cleanup_orphans()) Skipped 3 previous similar messages [ 5093.726870] Lustre: DEBUG MARKER: oleg140-client.virtnet: executing wait_import_state (FULL|IDLE) osc.lustre-OST0000-osc-ffff926948e39000.ost_server_uuid 50 [ 5094.353273] Lustre: DEBUG MARKER: osc.lustre-OST0000-osc-ffff926948e39000.ost_server_uuid in FULL state after 0 sec [ 5095.727543] Lustre: DEBUG MARKER: oleg140-client.virtnet: executing wait_import_state (FULL|IDLE) osc.lustre-OST0001-osc-ffff926948e39000.ost_server_uuid 50 [ 5096.270084] Lustre: DEBUG MARKER: osc.lustre-OST0001-osc-ffff926948e39000.ost_server_uuid in FULL state after 0 sec [ 5096.483865] LustreError: 310241:0:(osp_precreate.c:969:osp_precreate_cleanup_orphans()) lustre-OST0001-osc-MDT0001: cannot cleanup orphans: rc = -1 [ 5096.488623] LustreError: 310241:0:(osp_precreate.c:969:osp_precreate_cleanup_orphans()) Skipped 7 previous similar messages [ 5097.514743] Lustre: lustre-OST0001-osc-MDT0000: update sequence from 0x100010000 to 0x2c0000401 [ 5108.711147] Lustre: lustre-OST0001-osc-MDT0000: Connection to lustre-OST0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5108.714506] Lustre: lustre-OST0001: Not available for connect from 0@lo (stopping) [ 5108.715668] Lustre: Skipped 1 previous similar message [ 5108.719783] Lustre: Skipped 1 previous similar message [ 5114.096737] LustreError: 310987:0:(obd_class.h:479:obd_check_dev()) Device 25 not setup [ 5114.098740] LustreError: 310987:0:(obd_class.h:479:obd_check_dev()) Skipped 13 previous similar messages [ 5114.122997] Lustre: server umount lustre-OST0001 complete [ 5118.433156] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 5118.436132] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5118.948904] LustreError: 310694:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 5118.957072] LustreError: 310694:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 5 previous similar messages [ 5121.936813] Lustre: server umount lustre-OST0000 complete [ 5124.066556] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 5124.068891] LustreError: Skipped 1 previous similar message [ 5129.185872] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 5129.189293] Lustre: Skipped 8 previous similar messages [ 5129.413727] LustreError: 311390:0:(obd_class.h:479:obd_check_dev()) Device 28 not setup [ 5129.416569] LustreError: 311390:0:(obd_class.h:479:obd_check_dev()) Skipped 7 previous similar messages [ 5129.446724] Lustre: server umount lustre-MDT0000 complete [ 5130.792505] LustreError: 309685:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1763356715 with bad export cookie 13698912152071744554 [ 5130.796383] LustreError: MGC192.168.201.140@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 5130.798327] LustreError: 309685:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 5130.930846] Lustre: server umount lustre-MDT0001 complete [ 5134.789764] Lustre: DEBUG MARKER: oleg140-server.virtnet: executing unload_modules_local [ 5135.984726] Key type lgssc unregistered [ 5136.126479] LNet: 312122:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 5136.130969] LNetError: 312122:0:(acceptor.c:264:lnet_acceptor_remove_socket()) Interface ens2 not found [ 5136.140523] LNet: Removed LNI 192.168.201.140@tcp [ 5136.445665] Key type .llcrypt unregistered [ 5136.446546] Key type ._llcrypt unregistered [ 5145.502334] Key type ._llcrypt registered [ 5145.503729] Key type .llcrypt registered [ 5145.539942] Lustre: DEBUG MARKER: oleg140-server.virtnet: executing load_modules_local [ 5145.875787] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 5145.930694] alg: No test for adler32 (adler32-zlib) [ 5146.777734] Lustre: Lustre: Build Version: 2.16.61_41_g6a6a76f [ 5146.850346] LNet: Added LNI 192.168.201.140@tcp [8/256/0/180] [ 5148.424155] Key type lgssc registered [ 5148.789563] Lustre: Echo OBD driver; http://www.lustre.org/ [ 5151.327975] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 5151.335484] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 5152.500662] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 5153.679996] Lustre: DEBUG MARKER: oleg140-server.virtnet: executing set_default_debug -1 all [ 5155.388228] Lustre: DEBUG MARKER: oleg140-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 5157.524377] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 5157.665445] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 5159.005381] Lustre: DEBUG MARKER: oleg140-server.virtnet: executing set_default_debug -1 all [ 5162.349553] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 5162.491936] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 5163.494222] LustreError: 315053:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 5163.499972] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:36 to 0x280000401:65) [ 5163.502111] LustreError: 315053:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 2 previous similar messages [ 5164.598625] Lustre: DEBUG MARKER: oleg140-server.virtnet: executing set_default_debug -1 all [ 5166.454303] Lustre: DEBUG MARKER: oleg140-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 5167.593556] LustreError: 315053:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 5167.600912] LustreError: 315053:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 1 previous similar message [ 5168.905346] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 5168.983898] Lustre: lustre-OST0001: Imperative Recovery not enabled, recovery window 60-180 [ 5171.016256] Lustre: DEBUG MARKER: oleg140-server.virtnet: executing set_default_debug -1 all [ 5172.915261] Lustre: DEBUG MARKER: oleg140-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0001-osc-[-0-9a-f]*.ost_server_uuid [ 5174.252805] Lustre: lustre-OST0001: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x2c0000401:3 to 0x2c0000401:33) [ 5175.565859] Lustre: DEBUG MARKER: oleg140-server.virtnet: executing wait_import_state (FULL|IDLE) os[cp].lustre-OST0001-osc-MDT0001.ost_server_uuid 50 [ 5175.652618] Lustre: DEBUG MARKER: os[cp].lustre-OST0001-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [ 5256.166621] Lustre: lustre-OST0001-osc-MDT0000: Connection to lustre-OST0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5256.167643] LustreError: lustre-OST0001-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 5256.170652] Lustre: lustre-OST0001: Not available for connect from 0@lo (stopping) [ 5256.173213] Lustre: Skipped 1 previous similar message [ 5256.175166] LustreError: Skipped 1 previous similar message [ 5258.478609] LustreError: 316920:0:(obd_class.h:479:obd_check_dev()) Device 27 not setup [ 5258.498284] Lustre: server umount lustre-OST0001 complete [ 5260.768864] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 5260.771275] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5260.775415] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 5260.777080] Lustre: Skipped 1 previous similar message [ 5261.281917] LustreError: 315228:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 5261.286138] LustreError: 315228:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 1 previous similar message [ 5266.222116] LustreError: 317122:0:(obd_class.h:479:obd_check_dev()) Device 23 not setup [ 5266.225708] LustreError: 317122:0:(obd_class.h:479:obd_check_dev()) Skipped 3 previous similar messages [ 5266.262169] Lustre: server umount lustre-OST0000 complete [ 5270.497070] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 5270.499674] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5270.505058] Lustre: Skipped 1 previous similar message [ 5270.506881] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 5270.509702] Lustre: Skipped 2 previous similar messages [ 5273.601649] LustreError: 317323:0:(obd_class.h:479:obd_check_dev()) Device 10 not setup [ 5273.603710] LustreError: 317323:0:(obd_class.h:479:obd_check_dev()) Skipped 3 previous similar messages [ 5273.633717] Lustre: server umount lustre-MDT0000 complete [ 5274.931634] LustreError: 313480:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1763356860 with bad export cookie 14771831141774097391 [ 5274.937680] LustreError: MGC192.168.201.140@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 5275.064682] Lustre: server umount lustre-MDT0001 complete [ 5278.624298] Lustre: DEBUG MARKER: oleg140-server.virtnet: executing unload_modules_local [ 5279.637135] Key type lgssc unregistered [ 5279.772692] LNet: 318055:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 5279.775200] LNetError: 318055:0:(acceptor.c:264:lnet_acceptor_remove_socket()) Interface ens2 not found [ 5279.781550] LNet: Removed LNI 192.168.201.140@tcp [ 5280.094592] Key type .llcrypt unregistered [ 5280.095598] Key type ._llcrypt unregistered [ 5298.267776] Key type ._llcrypt registered [ 5298.269368] Key type .llcrypt registered [ 5298.314428] Lustre: DEBUG MARKER: oleg140-server.virtnet: executing load_modules_local [ 5298.678064] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 5298.724122] alg: No test for adler32 (adler32-zlib) [ 5299.579290] Lustre: Lustre: Build Version: 2.16.61_41_g6a6a76f [ 5299.678546] LNet: Added LNI 192.168.201.140@tcp [8/256/0/180] [ 5301.264222] Key type lgssc registered [ 5301.681897] Lustre: Echo OBD driver; http://www.lustre.org/ [ 5304.945518] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 5304.951295] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 5306.135321] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 5307.588580] Lustre: DEBUG MARKER: oleg140-server.virtnet: executing set_default_debug -1 all [ 5310.578721] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 5310.714561] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 5312.089027] Lustre: DEBUG MARKER: oleg140-server.virtnet: executing set_default_debug -1 all [ 5313.100277] Lustre: Modifying parameter general.debug_raw_pointers=Y in log params [ 5315.384204] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 5315.517727] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 5316.517830] LustreError: 321758:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 5316.522096] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:36 to 0x280000401:97) [ 5316.525883] LustreError: 321758:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 2 previous similar messages [ 5317.572297] Lustre: DEBUG MARKER: oleg140-server.virtnet: executing set_default_debug -1 all [ 5320.535851] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 5320.627282] Lustre: lustre-OST0001: Imperative Recovery not enabled, recovery window 60-180 [ 5322.683171] Lustre: DEBUG MARKER: oleg140-server.virtnet: executing set_default_debug -1 all [ 5325.808610] Lustre: lustre-OST0001: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x2c0000401:3 to 0x2c0000401:65) [ 5327.038465] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 5333.996065] Lustre: Setting parameter general.lod.*.mdt_hash=crush in log params [ 5346.275321] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5346.278160] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 5346.278568] Lustre: Skipped 1 previous similar message [ 5348.800244] LustreError: 323710:0:(obd_class.h:479:obd_check_dev()) Device 10 not setup [ 5348.831754] Lustre: server umount lustre-MDT0000 complete [ 5350.136293] LustreError: 320326:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1763356935 with bad export cookie 10865844314383978233 [ 5350.140174] LustreError: MGC192.168.201.140@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 5350.140831] LustreError: 320326:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 5350.218375] LustreError: 323912:0:(obd_class.h:479:obd_check_dev()) Device 18 not setup [ 5350.220289] LustreError: 323912:0:(obd_class.h:479:obd_check_dev()) Skipped 7 previous similar messages [ 5350.286111] Lustre: server umount lustre-MDT0001 complete [ 5361.843047] LustreError: 324112:0:(obd_class.h:479:obd_check_dev()) Device 23 not setup [ 5361.845027] LustreError: 324112:0:(obd_class.h:479:obd_check_dev()) Skipped 7 previous similar messages [ 5361.862297] Lustre: server umount lustre-OST0000 complete [ 5373.363959] LustreError: 324314:0:(obd_class.h:479:obd_check_dev()) Device 2 not setup [ 5373.366661] LustreError: 324314:0:(obd_class.h:479:obd_check_dev()) Skipped 3 previous similar messages [ 5373.407947] Lustre: server umount lustre-OST0001 complete [ 5379.037496] Lustre: DEBUG MARKER: oleg140-server.virtnet: executing load_modules_local [ 5382.692240] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 5382.849271] LustreError: 325489:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 5382.870922] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 5384.150384] Lustre: DEBUG MARKER: oleg140-server.virtnet: executing set_default_debug -1 all [ 5387.079933] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 5388.598957] Lustre: DEBUG MARKER: oleg140-server.virtnet: executing set_default_debug -1 all [ 5389.611681] Lustre: 326596:0:(mgs_llog.c:1346:mgs_modify_param()) MGS: modify general/debug_raw_pointers=Y (mode = 0) failed: rc = -17 [ 5391.892065] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 5392.025025] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 5392.029137] Lustre: Skipped 1 previous similar message [ 5393.063317] LustreError: 326956:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 5393.069397] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:36 to 0x280000401:129) [ 5393.981833] Lustre: DEBUG MARKER: oleg140-server.virtnet: executing set_default_debug -1 all [ 5397.033350] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 5398.193473] Lustre: lustre-OST0001: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x2c0000401:3 to 0x2c0000401:97) [ 5399.222269] Lustre: DEBUG MARKER: oleg140-server.virtnet: executing set_default_debug -1 all [ 5402.478296] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 5403.810103] Lustre: 328445:0:(mgs_llog.c:1346:mgs_modify_param()) MGS: modify general/lod.*.mdt_hash=crush (mode = 0) failed: rc = -17 [ 5412.833189] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 5412.836978] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5412.842928] Lustre: Skipped 2 previous similar messages [ 5412.845665] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 5412.847237] Lustre: Skipped 3 previous similar messages [ 5413.862922] Lustre: lustre-MDT0000-lwp-OST0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5413.863080] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 5413.869053] Lustre: Skipped 1 previous similar message [ 5413.870956] Lustre: Skipped 1 previous similar message [ 5415.938524] LustreError: 328549:0:(obd_class.h:479:obd_check_dev()) Device 10 not setup [ 5415.941637] LustreError: 328549:0:(obd_class.h:479:obd_check_dev()) Skipped 5 previous similar messages [ 5415.974928] Lustre: server umount lustre-MDT0000 complete [ 5417.233097] LustreError: 325468:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1763357002 with bad export cookie 10865844314383979920 [ 5417.236184] LustreError: 325468:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) Skipped 2 previous similar messages [ 5417.236463] LustreError: MGC192.168.201.140@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 5417.358821] Lustre: server umount lustre-MDT0001 complete [ 5428.150078] LustreError: 328952:0:(obd_class.h:479:obd_check_dev()) Device 23 not setup [ 5428.151891] LustreError: 328952:0:(obd_class.h:479:obd_check_dev()) Skipped 15 previous similar messages [ 5428.169747] Lustre: server umount lustre-OST0000 complete [ 5439.772265] Lustre: server umount lustre-OST0001 complete [ 5441.046234] Lustre: DEBUG MARKER: SKIP: conf-sanity test_114 skipping SLOW test 114 [ 5441.632784] Lustre: DEBUG MARKER: SKIP: conf-sanity test_115 skipping excluded test 115 [ 5442.255921] Lustre: DEBUG MARKER: == conf-sanity test 116: big size MDT support ============ 00:23:46 (1763357026) [ 5450.012943] Lustre: DEBUG MARKER: oleg140-server.virtnet: executing load_modules_local [ 5453.347868] loop0: detected capacity change from 0 to 1099511627776 [ 5453.526053] SGI XFS with ACLs, security attributes, quota, no debug enabled [ 5453.528517] *** VALIDATE xfs *** [ 5453.534834] XFS (loop0): Mounting V5 Filesystem [ 5453.543820] XFS (loop0): Ending clean mount [ 5453.972244] loop1: detected capacity change from 0 to 18691697672192 [ 5470.210893] LDISKFS-fs (loop1): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 5476.420828] Lustre: DEBUG MARKER: oleg140-server.virtnet: executing set_hostid [ 5479.913477] Lustre: DEBUG MARKER: oleg140-server.virtnet: executing load_modules_local [ 5483.497548] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 5485.729029] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 5487.935226] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 5490.254528] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 5492.578698] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 5492.610267] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 5492.712918] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 5492.725895] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 5492.768168] Lustre: lustre-MDT0000: new disk, initializing [ 5492.795652] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 5492.798664] Lustre: Skipped 1 previous similar message [ 5492.805721] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 5494.202538] Lustre: DEBUG MARKER: oleg140-server.virtnet: executing set_default_debug -1 all [ 5498.598085] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 5498.622809] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 5498.663393] Lustre: 333523:0:(mgs_llog.c:1346:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 5498.697576] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [ 5498.699189] Lustre: Skipped 1 previous similar message [ 5498.747170] Lustre: lustre-MDT0001: new disk, initializing [ 5498.801976] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [ 5498.804919] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 5500.153099] Lustre: DEBUG MARKER: oleg140-server.virtnet: executing set_default_debug -1 all [ 5503.436325] Lustre: DEBUG MARKER: oleg140-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 5504.725214] Lustre: DEBUG MARKER: oleg140-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 5507.139057] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 5507.171779] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 5507.293157] Lustre: lustre-OST0000: new disk, initializing [ 5507.296051] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 5508.605200] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [ 5508.608100] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 5508.649793] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 5509.280617] Lustre: DEBUG MARKER: oleg140-server.virtnet: executing set_default_debug -1 all [ 5512.459826] Lustre: DEBUG MARKER: oleg140-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 5513.955095] Lustre: DEBUG MARKER: oleg140-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 [ 5514.049670] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [ 5515.466537] Lustre: DEBUG MARKER: oleg140-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid 50 [ 5515.548679] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [ 5518.820269] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 5518.820755] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5518.822618] LustreError: Skipped 1 previous similar message [ 5518.828380] Lustre: Skipped 1 previous similar message [ 5518.830700] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 5518.834475] Lustre: Skipped 3 previous similar messages [ 5522.286120] LustreError: 336461:0:(obd_class.h:479:obd_check_dev()) Device 19 not setup [ 5522.288163] LustreError: 336461:0:(obd_class.h:479:obd_check_dev()) Skipped 9 previous similar messages [ 5522.330282] Lustre: server umount lustre-OST0000 complete [ 5523.937461] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5523.937979] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 5523.940548] Lustre: Skipped 1 previous similar message [ 5529.757162] Lustre: server umount lustre-MDT0000 complete [ 5531.106570] LustreError: 333514:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1763357116 with bad export cookie 10865844314383981474 [ 5531.110252] LustreError: MGC192.168.201.140@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 5531.112122] LustreError: 333514:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 5531.259375] Lustre: server umount lustre-MDT0001 complete [ 5534.210302] XFS (loop0): Unmounting Filesystem [ 5536.009924] Lustre: DEBUG MARKER: == conf-sanity test 117: lctl get_param return errors properly ========================================================== 00:25:20 (1763357120) [ 5538.446494] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 5538.623385] LustreError: 337680:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 5538.652580] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 5538.655329] Lustre: Skipped 2 previous similar messages [ 5540.102127] Lustre: DEBUG MARKER: oleg140-server.virtnet: executing set_default_debug -1 all [ 5543.112918] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 5544.672841] Lustre: DEBUG MARKER: oleg140-server.virtnet: executing set_default_debug -1 all [ 5546.624779] Lustre: DEBUG MARKER: oleg140-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 5547.888910] Lustre: DEBUG MARKER: oleg140-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 5550.279173] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 5552.368091] Lustre: DEBUG MARKER: oleg140-server.virtnet: executing set_default_debug -1 all [ 5554.224695] Lustre: DEBUG MARKER: oleg140-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 5568.995309] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 5568.995406] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5568.999080] LustreError: Skipped 1 previous similar message [ 5568.999906] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 5569.004360] Lustre: Skipped 1 previous similar message [ 5569.006868] Lustre: Skipped 3 previous similar messages [ 5570.094143] LustreError: 340044:0:(obd_class.h:479:obd_check_dev()) Device 21 not setup [ 5570.097215] LustreError: 340044:0:(obd_class.h:479:obd_check_dev()) Skipped 17 previous similar messages [ 5570.137356] Lustre: server umount lustre-OST0000 complete [ 5573.605177] Lustre: server umount lustre-MDT0000 complete [ 5574.114749] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 5574.116302] LustreError: 337679:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 5574.125101] LustreError: 337679:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 1 previous similar message [ 5574.926283] LustreError: 337660:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1763357160 with bad export cookie 10865844314383982356 [ 5574.930186] LustreError: MGC192.168.201.140@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 5574.931452] LustreError: 337660:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 5575.055699] Lustre: server umount lustre-MDT0001 complete [ 5579.032389] Lustre: DEBUG MARKER: oleg140-server.virtnet: executing unload_modules_local [ 5580.231604] Key type lgssc unregistered [ 5580.384706] LNet: 340976:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 5580.389049] LNetError: 340976:0:(acceptor.c:264:lnet_acceptor_remove_socket()) Interface ens2 not found [ 5580.398481] LNet: Removed LNI 192.168.201.140@tcp [ 5580.777161] Key type .llcrypt unregistered [ 5580.778773] Key type ._llcrypt unregistered [ 5591.504110] Key type ._llcrypt registered [ 5591.505865] Key type .llcrypt registered [ 5591.549949] Lustre: DEBUG MARKER: oleg140-server.virtnet: executing load_modules_local [ 5591.898687] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 5591.948090] alg: No test for adler32 (adler32-zlib) [ 5592.804210] Lustre: Lustre: Build Version: 2.16.61_41_g6a6a76f [ 5592.889406] LNet: Added LNI 192.168.201.140@tcp [8/256/0/180] [ 5594.472188] Key type lgssc registered [ 5594.867280] Lustre: Echo OBD driver; http://www.lustre.org/ [ 5597.628518] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 5597.634587] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 5598.804679] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 5600.158073] Lustre: DEBUG MARKER: oleg140-server.virtnet: executing set_default_debug -1 all [ 5603.172710] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 5603.302954] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 5604.685275] Lustre: DEBUG MARKER: oleg140-server.virtnet: executing set_default_debug -1 all [ 5606.605385] Lustre: DEBUG MARKER: oleg140-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 5607.902476] Lustre: DEBUG MARKER: oleg140-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 5610.240769] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 5610.370053] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 5612.441247] Lustre: DEBUG MARKER: oleg140-server.virtnet: executing set_default_debug -1 all [ 5614.286119] Lustre: DEBUG MARKER: oleg140-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 5628.903321] Lustre: lustre-MDT0001-lwp-OST0000: Connection to lustre-MDT0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5628.903730] LustreError: lustre-MDT0001-osp-MDT0000: operation mds_statfs to node 0@lo failed: rc = -107 [ 5628.911152] Lustre: lustre-MDT0001: Not available for connect from 0@lo (stopping) [ 5628.915852] Lustre: Skipped 1 previous similar message [ 5629.896656] Lustre: lustre-MDT0001: Not available for connect from 192.168.201.40@tcp (stopping) [ 5633.538910] LustreError: 344818:0:(obd_class.h:479:obd_check_dev()) Device 16 not setup [ 5633.559594] Lustre: server umount lustre-MDT0001 complete [ 5634.018377] LustreError: 342501:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 5634.022276] LustreError: 342501:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 3 previous similar messages [ 5634.560191] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 5635.008099] LustreError: 344661:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0001: not available for connect from 192.168.201.40@tcp (no target). If you are running an HA pair check that the target is mounted on the other server. [ 5637.186202] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 5637.224933] Lustre: MGS: Regenerating lustre-MDT0001 log by user request: rc = 0 [ 5637.226584] Lustre: Found index 1 for lustre-MDT0001, updating log [ 5637.237396] Lustre: 342491:0:(mgs_llog.c:1346:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 5637.352856] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 5638.755836] Lustre: DEBUG MARKER: oleg140-server.virtnet: executing set_default_debug -1 all [ 5642.729860] Lustre: lustre-MDT0001-lwp-OST0000: Connection restored to 0@lo (at 0@lo) [ 5646.822239] LustreError: 345997:0:(obd_mount.c:188:lustre_start_simple()) lustre-MDT0001-lwp-OST0000 attach error -17 [ 5646.824522] LustreError: 345997:0:(tgt_mount.c:669:lustre_lwp_setup()) lustre-MDT0001-lwp-OST0000: setup up failed: rc -17 [ 5646.827084] LustreError: 342493:0:(mgc_request.c:614:do_requeue()) failed processing log: -17 [ 5646.844719] LustreError: 346000:0:(obd_config.c:1997:class_config_llog_handler()) MGC192.168.201.140@tcp: cfg command failed: rc = -17 [ 5646.847254] Lustre: cmd=cf001 0:lustre-MDT0001-osp-MDT0000 1:osp 2:lustre-MDT0000-mdtlov_UUID [ 5646.847254] [ 5942.216399] Lustre: lustre-MDT0001: Not available for connect from 192.168.201.40@tcp (stopping) [ 5942.757151] Lustre: lustre-MDT0001-lwp-OST0000: Connection to lustre-MDT0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5942.763149] Lustre: Skipped 2 previous similar messages [ 5946.942372] LustreError: 357137:0:(obd_class.h:479:obd_check_dev()) Device 16 not setup [ 5946.944312] LustreError: 357137:0:(obd_class.h:479:obd_check_dev()) Skipped 5 previous similar messages [ 5946.974708] Lustre: server umount lustre-MDT0001 complete [ 5947.327838] LustreError: 342497:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0001: not available for connect from 192.168.201.40@tcp (no target). If you are running an HA pair check that the target is mounted on the other server. [ 5948.199802] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 5951.457340] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 5951.498666] Lustre: MGS: Regenerating lustre-MDT0001 log by user request: rc = 0 [ 5951.500647] Lustre: Found index 1 for lustre-MDT0001, updating log [ 5951.515502] Lustre: 342489:0:(mgs_llog.c:1346:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 5951.649921] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 5953.124781] Lustre: DEBUG MARKER: oleg140-server.virtnet: executing set_default_debug -1 all [ 5957.096693] Lustre: lustre-MDT0001-lwp-OST0000: Connection restored to 0@lo (at 0@lo) [ 5961.711545] LustreError: 358362:0:(obd_mount.c:188:lustre_start_simple()) lustre-MDT0001-lwp-OST0000 attach error -17 [ 5961.717242] LustreError: 358362:0:(tgt_mount.c:669:lustre_lwp_setup()) lustre-MDT0001-lwp-OST0000: setup up failed: rc -17 [ 5961.722831] LustreError: 342493:0:(mgc_request.c:614:do_requeue()) failed processing log: -17 [ 5961.726394] LustreError: 342493:0:(mgc_request.c:614:do_requeue()) Skipped 1 previous similar message [ 5961.747722] LustreError: 358365:0:(obd_config.c:689:class_setup()) Device 8 already setup (type osp) [ 5961.750276] LustreError: 358365:0:(obd_config.c:1997:class_config_llog_handler()) MGC192.168.201.140@tcp: cfg command failed: rc = -17 [ 5961.754886] Lustre: cmd=cf003 0:lustre-MDT0001-osp-MDT0000 1:lustre-MDT0001_UUID 2:192.168.201.140@tcp [ 5961.754886] [ 6259.171508] LustreError: lustre-MDT0001-osp-MDT0000: operation mds_statfs to node 0@lo failed: rc = -107 [ 6259.172519] Lustre: lustre-MDT0001-lwp-OST0000: Connection to lustre-MDT0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 6259.173200] Lustre: lustre-MDT0001: Not available for connect from 0@lo (stopping) [ 6259.173204] Lustre: Skipped 2 previous similar messages [ 6259.184233] Lustre: Skipped 1 previous similar message [ 6260.936561] LustreError: 369249:0:(obd_class.h:479:obd_check_dev()) Device 16 not setup [ 6260.939262] LustreError: 369249:0:(obd_class.h:479:obd_check_dev()) Skipped 5 previous similar messages [ 6260.970330] Lustre: server umount lustre-MDT0001 complete [ 6262.078387] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 6264.290218] LustreError: 342501:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 6264.296059] LustreError: 342501:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 3 previous similar messages [ 6264.864571] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 6264.903418] Lustre: MGS: Regenerating lustre-MDT0001 log by user request: rc = 0 [ 6264.906040] Lustre: Found index 1 for lustre-MDT0001, updating log [ 6264.920598] Lustre: 342489:0:(mgs_llog.c:1346:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 6265.052024] Lustre: lustre-MDT0001: Imperative Recovery enabled, recovery window shrunk from 60-180 down to 60-180 [ 6266.505852] Lustre: DEBUG MARKER: oleg140-server.virtnet: executing set_default_debug -1 all [ 6270.444316] Lustre: lustre-MDT0001-lwp-OST0000: Connection restored to 0@lo (at 0@lo) [ 6270.450925] LustreError: 370294:0:(obd_mount.c:188:lustre_start_simple()) lustre-MDT0001-lwp-OST0000 attach error -17 [ 6270.453201] LustreError: 370294:0:(tgt_mount.c:669:lustre_lwp_setup()) lustre-MDT0001-lwp-OST0000: setup up failed: rc -17 [ 6270.455586] LustreError: 342493:0:(mgc_request.c:614:do_requeue()) failed processing log: -17 [ 6270.457522] LustreError: 342493:0:(mgc_request.c:614:do_requeue()) Skipped 1 previous similar message [ 6270.476987] LustreError: 370297:0:(lod_lov.c:207:lod_add_device()) lustre-MDT0001-osp-MDT0000: cannot connect to next dev lustre-MDT0001_UUID (-114) [ 6270.481398] LustreError: 370297:0:(obd_config.c:1997:class_config_llog_handler()) MGC192.168.201.140@tcp: cfg command failed: rc = -114 [ 6270.484436] Lustre: cmd=cf014 0:lustre-MDT0000-mdtlov 1:lustre-MDT0001_UUID 2:1 3:1 [ 6270.484436] [ 6582.755076] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 6582.756504] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 6582.760587] Lustre: Skipped 2 previous similar messages [ 6582.760926] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 6582.762138] Lustre: Skipped 4 previous similar messages [ 6587.200227] LustreError: 381933:0:(obd_class.h:479:obd_check_dev()) Device 9 not setup [ 6587.202548] LustreError: 381933:0:(obd_class.h:479:obd_check_dev()) Skipped 5 previous similar messages [ 6587.231351] Lustre: server umount lustre-MDT0000 complete [ 6587.873312] LustreError: 344117:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 6587.877682] LustreError: 344117:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 3 previous similar messages [ 6588.424373] LustreError: 344119:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1763358173 with bad export cookie 17785285857417959120 [ 6588.427995] LustreError: MGC192.168.201.140@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 6588.427994] LustreError: 344119:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 6592.993421] Lustre: lustre-MDT0001-lwp-OST0000: Connection to lustre-MDT0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 6592.997325] Lustre: lustre-MDT0001: Not available for connect from 0@lo (stopping) [ 6594.567483] LustreError: 382134:0:(obd_class.h:479:obd_check_dev()) Device 16 not setup [ 6594.571137] LustreError: 382134:0:(obd_class.h:479:obd_check_dev()) Skipped 5 previous similar messages [ 6594.634260] Lustre: server umount lustre-MDT0001 complete [ 6595.978919] Lustre: server umount lustre-OST0000 complete [ 6598.965602] Lustre: DEBUG MARKER: oleg140-server.virtnet: executing set_hostid [ 6602.037735] Lustre: DEBUG MARKER: oleg140-server.virtnet: executing load_modules_local [ 6605.588503] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 6607.678360] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 6609.834712] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 6612.128149] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 6614.407501] Lustre: DEBUG MARKER: == conf-sanity test 120: cross-target rename should not create bad symlinks ========================================================== 00:43:18 (1763358198) [ 6616.732172] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 6616.759344] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 6616.849334] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 6616.862278] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 6616.900114] Lustre: lustre-MDT0000: new disk, initializing [ 6616.925850] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 6616.932606] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 6618.233486] Lustre: DEBUG MARKER: oleg140-server.virtnet: executing set_default_debug -1 all [ 6622.291739] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 6622.314088] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 6622.347364] Lustre: 384865:0:(mgs_llog.c:1346:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 6622.368878] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [ 6622.370566] Lustre: Skipped 1 previous similar message [ 6622.425472] Lustre: lustre-MDT0001: new disk, initializing [ 6622.461449] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 6622.472451] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [ 6622.475475] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 6623.749624] Lustre: DEBUG MARKER: oleg140-server.virtnet: executing set_default_debug -1 all [ 6626.906922] Lustre: DEBUG MARKER: oleg140-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 6628.160052] Lustre: DEBUG MARKER: oleg140-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 6630.318599] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 6630.343960] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 6630.449584] Lustre: lustre-OST0000: new disk, initializing [ 6630.451441] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 6630.483814] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 6632.304065] Lustre: DEBUG MARKER: oleg140-server.virtnet: executing set_default_debug -1 all [ 6632.763148] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [ 6632.765851] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 6632.803394] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 6635.347340] Lustre: DEBUG MARKER: oleg140-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 6638.051318] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 6638.051388] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 6638.052459] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 6638.052464] Lustre: Skipped 1 previous similar message [ 6638.054833] Lustre: Skipped 1 previous similar message [ 6644.093751] LustreError: 387312:0:(obd_class.h:479:obd_check_dev()) Device 22 not setup [ 6644.095373] LustreError: 387312:0:(obd_class.h:479:obd_check_dev()) Skipped 11 previous similar messages [ 6644.127849] Lustre: server umount lustre-MDT0000 complete [ 6645.396424] LustreError: 385646:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1763358230 with bad export cookie 17785285857417961066 [ 6645.401100] LustreError: 385658:0:(osp_object.c:617:osp_attr_get()) lustre-MDT0000-osp-MDT0001: osp_attr_get update error [0x200000401:0x2:0x0]: rc = -5 [ 6645.402546] LustreError: MGC192.168.201.140@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 6645.533646] Lustre: server umount lustre-MDT0001 complete [ 6656.028518] Lustre: server umount lustre-OST0000 complete [ 6665.275382] Lustre: DEBUG MARKER: oleg140-server.virtnet: executing set_hostid [ 6668.608647] Lustre: DEBUG MARKER: oleg140-server.virtnet: executing load_modules_local [ 6672.019080] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 6674.226141] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 6676.231912] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 6678.404334] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 6678.989242] Lustre: DEBUG MARKER: == conf-sanity test complete, duration 6474 sec ========== 00:44:23 (1763358263) [ 6679.543123] Lustre: DEBUG MARKER: === conf-sanity: start cleanup 00:44:24 (1763358264) === [ 6680.633750] Lustre: DEBUG MARKER: === conf-sanity: finish cleanup 00:44:25 (1763358265) === [ 6693.841032] Lustre: DEBUG MARKER: oleg140-server.virtnet: executing unload_modules_local [ 6694.955093] Key type lgssc unregistered [ 6695.083401] LNet: 392814:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 6695.086240] LNetError: 392814:0:(acceptor.c:264:lnet_acceptor_remove_socket()) Interface ens2 not found [ 6695.096396] LNet: Removed LNI 192.168.201.140@tcp [ 6695.440358] Key type .llcrypt unregistered [ 6695.441441] Key type ._llcrypt unregistered