[ 0.000000] Linux version 4.18.0rh8.10-debug (green@maintenance) (gcc version 8.5.0 20210514 (Red Hat 8.5.0-26) (GCC)) #2 SMP Mon Jul 14 01:24:22 EDT 2025 [ 0.000000] Command line: rd.shell root=nbd:192.168.200.253:rocky8.10:ext4:ro:-p,-b4096 ro crashkernel=256M panic=1 nomodeset ipmtu=9000 ip=dhcp rd.neednet=1 init_on_free=off mitigations=off console=ttyS1,115200 audit=0 [ 0.000000] x86/fpu: Supporting XSAVE feature 0x001: 'x87 floating point registers' [ 0.000000] x86/fpu: Supporting XSAVE feature 0x002: 'SSE registers' [ 0.000000] x86/fpu: Supporting XSAVE feature 0x004: 'AVX registers' [ 0.000000] x86/fpu: xstate_offset[2]: 576, xstate_sizes[2]: 256 [ 0.000000] x86/fpu: Enabled xstate features 0x7, context size is 832 bytes, using 'standard' format. [ 0.000000] signal: max sigframe size: 1776 [ 0.000000] BIOS-provided physical RAM map: [ 0.000000] BIOS-e820: [mem 0x0000000000000000-0x000000000009fbff] usable [ 0.000000] BIOS-e820: [mem 0x000000000009fc00-0x000000000009ffff] reserved [ 0.000000] BIOS-e820: [mem 0x00000000000f0000-0x00000000000fffff] reserved [ 0.000000] BIOS-e820: [mem 0x0000000000100000-0x00000000bffcdfff] usable [ 0.000000] BIOS-e820: [mem 0x00000000bffce000-0x00000000bfffffff] reserved [ 0.000000] BIOS-e820: [mem 0x00000000feffc000-0x00000000feffffff] reserved [ 0.000000] BIOS-e820: [mem 0x00000000fffc0000-0x00000000ffffffff] reserved [ 0.000000] BIOS-e820: [mem 0x0000000100000000-0x0000000146dfffff] usable [ 0.000000] NX (Execute Disable) protection: active [ 0.000000] SMBIOS 3.0.0 present. [ 0.000000] DMI: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.16.3-2.fc40 04/01/2014 [ 0.000000] Hypervisor detected: KVM [ 0.000000] kvm-clock: Using msrs 4b564d01 and 4b564d00 [ 0.000000] kvm-clock: using sched offset of 466911122 cycles [ 0.000000] clocksource: kvm-clock: mask: 0xffffffffffffffff max_cycles: 0x1cd42e4dffb, max_idle_ns: 881590591483 ns [ 0.000000] tsc: Detected 2399.998 MHz processor [ 0.000000] last_pfn = 0x146e00 max_arch_pfn = 0x400000000 [ 0.000000] x86/PAT: Configuration [0-7]: WB WC UC- UC WB WP UC- WT [ 0.000000] last_pfn = 0xbffce max_arch_pfn = 0x400000000 [ 0.000000] found SMP MP-table at [mem 0x000f53f0-0x000f53ff] [ 0.000000] RAMDISK: [mem 0xbcc54000-0xbffbffff] [ 0.000000] ACPI: Early table checksum verification disabled [ 0.000000] ACPI: RSDP 0x00000000000F5200 000014 (v00 BOCHS ) [ 0.000000] ACPI: RSDT 0x00000000BFFE1D87 000034 (v01 BOCHS BXPC 00000001 BXPC 00000001) [ 0.000000] ACPI: FACP 0x00000000BFFE1C23 000074 (v01 BOCHS BXPC 00000001 BXPC 00000001) [ 0.000000] ACPI: DSDT 0x00000000BFFE0040 001BE3 (v01 BOCHS BXPC 00000001 BXPC 00000001) [ 0.000000] ACPI: FACS 0x00000000BFFE0000 000040 [ 0.000000] ACPI: APIC 0x00000000BFFE1C97 000090 (v03 BOCHS BXPC 00000001 BXPC 00000001) [ 0.000000] ACPI: HPET 0x00000000BFFE1D27 000038 (v01 BOCHS BXPC 00000001 BXPC 00000001) [ 0.000000] ACPI: WAET 0x00000000BFFE1D5F 000028 (v01 BOCHS BXPC 00000001 BXPC 00000001) [ 0.000000] ACPI: Reserving FACP table memory at [mem 0xbffe1c23-0xbffe1c96] [ 0.000000] ACPI: Reserving DSDT table memory at [mem 0xbffe0040-0xbffe1c22] [ 0.000000] ACPI: Reserving FACS table memory at [mem 0xbffe0000-0xbffe003f] [ 0.000000] ACPI: Reserving APIC table memory at [mem 0xbffe1c97-0xbffe1d26] [ 0.000000] ACPI: Reserving HPET table memory at [mem 0xbffe1d27-0xbffe1d5e] [ 0.000000] ACPI: Reserving WAET table memory at [mem 0xbffe1d5f-0xbffe1d86] [ 0.000000] No NUMA configuration found [ 0.000000] Faking a node at [mem 0x0000000000000000-0x0000000146dfffff] [ 0.000000] NODE_DATA(0) allocated [mem 0x1465a3000-0x1465cdfff] [ 0.000000] Reserving 256MB of memory at 2752MB for crashkernel (System RAM: 4205MB) [ 0.000000] Zone ranges: [ 0.000000] DMA [mem 0x0000000000001000-0x0000000000ffffff] [ 0.000000] DMA32 [mem 0x0000000001000000-0x00000000ffffffff] [ 0.000000] Normal [mem 0x0000000100000000-0x0000000146dfffff] [ 0.000000] Device empty [ 0.000000] Movable zone start for each node [ 0.000000] Early memory node ranges [ 0.000000] node 0: [mem 0x0000000000001000-0x000000000009efff] [ 0.000000] node 0: [mem 0x0000000000100000-0x00000000bffcdfff] [ 0.000000] node 0: [mem 0x0000000100000000-0x0000000146dfffff] [ 0.000000] Zeroed struct page in unavailable ranges: 4756 pages [ 0.000000] Initmem setup node 0 [mem 0x0000000000001000-0x0000000146dfffff] [ 0.000000] ACPI: PM-Timer IO Port: 0x608 [ 0.000000] ACPI: LAPIC_NMI (acpi_id[0xff] dfl dfl lint[0x1]) [ 0.000000] IOAPIC[0]: apic_id 0, version 17, address 0xfec00000, GSI 0-23 [ 0.000000] ACPI: INT_SRC_OVR (bus 0 bus_irq 0 global_irq 2 dfl dfl) [ 0.000000] ACPI: INT_SRC_OVR (bus 0 bus_irq 5 global_irq 5 high level) [ 0.000000] ACPI: INT_SRC_OVR (bus 0 bus_irq 9 global_irq 9 high level) [ 0.000000] ACPI: INT_SRC_OVR (bus 0 bus_irq 10 global_irq 10 high level) [ 0.000000] ACPI: INT_SRC_OVR (bus 0 bus_irq 11 global_irq 11 high level) [ 0.000000] Using ACPI (MADT) for SMP configuration information [ 0.000000] ACPI: HPET id: 0x8086a201 base: 0xfed00000 [ 0.000000] TSC deadline timer available [ 0.000000] smpboot: Allowing 4 CPUs, 0 hotplug CPUs [ 0.000000] kvm-guest: KVM setup pv remote TLB flush [ 0.000000] kvm-guest: setup PV sched yield [ 0.000000] PM: Registered nosave memory: [mem 0x00000000-0x00000fff] [ 0.000000] PM: Registered nosave memory: [mem 0x0009f000-0x0009ffff] [ 0.000000] PM: Registered nosave memory: [mem 0x000a0000-0x000effff] [ 0.000000] PM: Registered nosave memory: [mem 0x000f0000-0x000fffff] [ 0.000000] PM: Registered nosave memory: [mem 0xbffce000-0xbfffffff] [ 0.000000] PM: Registered nosave memory: [mem 0xc0000000-0xfeffbfff] [ 0.000000] PM: Registered nosave memory: [mem 0xfeffc000-0xfeffffff] [ 0.000000] PM: Registered nosave memory: [mem 0xff000000-0xfffbffff] [ 0.000000] PM: Registered nosave memory: [mem 0xfffc0000-0xffffffff] [ 0.000000] [mem 0xc0000000-0xfeffbfff] available for PCI devices [ 0.000000] Booting paravirtualized kernel on KVM [ 0.000000] clocksource: refined-jiffies: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 1910969940391419 ns [ 0.000000] setup_percpu: NR_CPUS:8192 nr_cpumask_bits:4 nr_cpu_ids:4 nr_node_ids:1 [ 0.000000] percpu: Embedded 63 pages/cpu s221184 r8192 d28672 u524288 [ 0.000000] kvm-guest: PV spinlocks enabled [ 0.000000] PV qspinlock hash table entries: 256 (order: 0, 4096 bytes, linear) [ 0.000000] Built 1 zonelists, mobility grouping on. Total pages: 1059606 [ 0.000000] Policy zone: Normal [ 0.000000] Kernel command line: rd.shell root=nbd:192.168.200.253:rocky8.10:ext4:ro:-p,-b4096 ro crashkernel=256M panic=1 nomodeset ipmtu=9000 ip=dhcp rd.neednet=1 init_on_free=off mitigations=off console=ttyS1,115200 audit=0 [ 0.000000] Specific versions of hardware are certified with Red Hat Enterprise Linux 8. Please see the list of hardware certified with Red Hat Enterprise Linux 8 at https://catalog.redhat.com. [ 0.000000] audit: disabled (until reboot) [ 0.000000] software IO TLB: area num 4. [ 0.000000] Memory: 2829652K/4306352K available (18435K kernel code, 11221K rwdata, 7248K rodata, 2908K init, 18040K bss, 524580K reserved, 0K cma-reserved) [ 0.000000] SLUB: HWalign=64, Order=0-3, MinObjects=0, CPUs=4, Nodes=1 [ 0.000000] kmemleak: Kernel memory leak detector disabled [ 0.000000] ftrace: allocating 41240 entries in 162 pages [ 0.000000] ftrace: allocated 162 pages with 3 groups [ 0.000000] rcu: Hierarchical RCU implementation. [ 0.000000] rcu: RCU event tracing is enabled. [ 0.000000] rcu: RCU restricting CPUs from NR_CPUS=8192 to nr_cpu_ids=4. [ 0.000000] rcu: RCU callback double-/use-after-free debug enabled. [ 0.000000] Rude variant of Tasks RCU enabled. [ 0.000000] Tracing variant of Tasks RCU enabled. [ 0.000000] rcu: RCU calculated value of scheduler-enlistment delay is 100 jiffies. [ 0.000000] rcu: Adjusting geometry for rcu_fanout_leaf=16, nr_cpu_ids=4 [ 0.000000] NR_IRQS: 524544, nr_irqs: 456, preallocated irqs: 16 [ 0.000000] random: get_random_bytes called from start_kernel+0x622/0x9a8 with crng_init=0 [ 0.001000] Console: colour *CGA 80x25 [ 0.001000] printk: console [ttyS1] enabled [ 0.001000] ACPI: Core revision 20220331 [ 0.001000] clocksource: hpet: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 19112604467 ns [ 0.001011] APIC: Switch to symmetric I/O mode setup [ 0.002358] x2apic enabled [ 0.003006] Switched APIC routing to physical x2apic. [ 0.004013] kvm-guest: setup PV IPIs [ 0.007322] ..TIMER: vector=0x30 apic1=0 pin1=2 apic2=-1 pin2=-1 [ 0.008000] clocksource: tsc-early: mask: 0xffffffffffffffff max_cycles: 0x229835b7123, max_idle_ns: 440795242976 ns [ 0.008028] Calibrating delay loop (skipped) preset value.. 4799.99 BogoMIPS (lpj=2399998) [ 0.009000] pid_max: default: 32768 minimum: 301 [ 0.010066] LSM: Security Framework initializing [ 0.011049] Yama: becoming mindful. [ 0.012000] SELinux: Initializing. [ 0.013058] *** VALIDATE selinux *** [ 0.016000] Dentry cache hash table entries: 1048576 (order: 11, 8388608 bytes, vmalloc) [ 0.016000] Inode-cache hash table entries: 524288 (order: 10, 4194304 bytes, vmalloc) [ 0.016139] Mount-cache hash table entries: 16384 (order: 5, 131072 bytes, vmalloc) [ 0.018074] Mountpoint-cache hash table entries: 16384 (order: 5, 131072 bytes, vmalloc) [ 0.019125] *** VALIDATE tmpfs *** [ 0.020000] *** VALIDATE proc *** [ 0.020000] *** VALIDATE cgroup *** [ 0.020000] *** VALIDATE cgroup2 *** [ 0.021166] x86/cpu: User Mode Instruction Prevention (UMIP) activated [ 0.022155] Last level iTLB entries: 4KB 0, 2MB 0, 4MB 0 [ 0.023008] Last level dTLB entries: 4KB 0, 2MB 0, 4MB 0, 1GB 0 [ 0.024029] Spectre V2 : User space: Vulnerable [ 0.025008] Speculative Store Bypass: Vulnerable [ 0.028422] debug: unmapping init [mem 0xffffffffb6859000-0xffffffffb6860fff] [ 0.031257] smpboot: CPU0: Intel(R) Xeon(R) CPU E5-2695 v2 @ 2.40GHz (family: 0x6, model: 0x3e, stepping: 0x4) [ 0.032760] Performance Events: IvyBridge events, full-width counters, Intel PMU driver. [ 0.033068] ... version: 2 [ 0.034011] ... bit width: 48 [ 0.035013] ... generic registers: 4 [ 0.036012] ... value mask: 0000ffffffffffff [ 0.037014] ... max period: 00007fffffffffff [ 0.038010] ... fixed-purpose events: 3 [ 0.039010] ... event mask: 000000070000000f [ 0.041249] rcu: Hierarchical SRCU implementation. [ 0.043573] smp: Bringing up secondary CPUs ... [ 0.044564] x86: Booting SMP configuration: [ 0.045027] .... node #0, CPUs: #1 #2 #3 [ 0.049622] smp: Brought up 1 node, 4 CPUs [ 0.051014] smpboot: Max logical packages: 1 [ 0.052016] smpboot: Total of 4 processors activated (19199.98 BogoMIPS) [ 0.210063] node 0 deferred pages initialised in 155ms [ 0.214130] devtmpfs: initialized [ 0.217582] x86/mm: Memory block size: 128MB [ 0.224166] gcov: version magic: 0x41383552 [ 0.227435] clocksource: jiffies: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 1911260446275000 ns [ 0.232105] futex hash table entries: 1024 (order: 4, 65536 bytes, vmalloc) [ 0.235318] pinctrl core: initialized pinctrl subsystem [ 0.237156] [ 0.237606] ************************************************************* [ 0.240124] ** NOTICE NOTICE NOTICE NOTICE NOTICE NOTICE NOTICE ** [ 0.244021] ** ** [ 0.247018] ** IOMMU DebugFS SUPPORT HAS BEEN ENABLED IN THIS KERNEL ** [ 0.250023] ** ** [ 0.253016] ** This means that this kernel is built to expose internal ** [ 0.256015] ** IOMMU data structures, which may compromise security on ** [ 0.259016] ** your system. ** [ 0.261014] ** ** [ 0.263019] ** If you see this message and you are not debugging the ** [ 0.266017] ** kernel, report this immediately to your vendor! ** [ 0.269015] ** ** [ 0.272016] ** NOTICE NOTICE NOTICE NOTICE NOTICE NOTICE NOTICE ** [ 0.275019] ************************************************************* [ 0.279407] NET: Registered protocol family 16 [ 0.281952] DMA: preallocated 512 KiB GFP_KERNEL pool for atomic allocations [ 0.286088] DMA: preallocated 512 KiB GFP_KERNEL|GFP_DMA pool for atomic allocations [ 0.291101] DMA: preallocated 512 KiB GFP_KERNEL|GFP_DMA32 pool for atomic allocations [ 0.296261] cpuidle: using governor menu [ 0.299020] acpiphp: ACPI Hot Plug PCI Controller Driver version: 0.5 [ 0.303073] PCI: Using configuration type 1 for base access [ 0.305162] core: PMU erratum BJ122, BV98, HSD29 worked around, HT is on [ 0.315142] HugeTLB registered 1.00 GiB page size, pre-allocated 0 pages [ 0.316025] HugeTLB registered 2.00 MiB page size, pre-allocated 0 pages [ 0.317186] cryptd: max_cpu_qlen set to 1000 [ 0.321023] ACPI: Added _OSI(Module Device) [ 0.323017] ACPI: Added _OSI(Processor Device) [ 0.325016] ACPI: Added _OSI(3.0 _SCP Extensions) [ 0.326016] ACPI: Added _OSI(Processor Aggregator Device) [ 0.332402] ACPI: 1 ACPI AML tables successfully acquired and loaded [ 0.341333] ACPI: Interpreter enabled [ 0.343079] ACPI: PM: (supports S0 S3 S4 S5) [ 0.345023] ACPI: Using IOAPIC for interrupt routing [ 0.347121] PCI: Using host bridge windows from ACPI; if necessary, use "pci=nocrs" and report a bug [ 0.350560] ACPI: Enabled 2 GPEs in block 00 to 0F [ 0.359754] ACPI: PCI Root Bridge [PCI0] (domain 0000 [bus 00-ff]) [ 0.362038] acpi PNP0A03:00: _OSC: OS supports [ASPM ClockPM Segments MSI HPX-Type3] [ 0.365022] acpi PNP0A03:00: _OSC: not requesting OS control; OS requires [ExtendedConfig ASPM ClockPM MSI] [ 0.368142] acpi PNP0A03:00: fail to add MMCONFIG information, can't access extended PCI configuration space under this bridge. [ 0.374220] acpiphp: Slot [2] registered [ 0.375160] acpiphp: Slot [5] registered [ 0.377161] acpiphp: Slot [6] registered [ 0.379140] acpiphp: Slot [7] registered [ 0.380120] acpiphp: Slot [8] registered [ 0.382101] acpiphp: Slot [9] registered [ 0.383127] acpiphp: Slot [10] registered [ 0.385145] acpiphp: Slot [3] registered [ 0.386248] acpiphp: Slot [4] registered [ 0.388099] acpiphp: Slot [11] registered [ 0.389128] acpiphp: Slot [12] registered [ 0.391088] acpiphp: Slot [13] registered [ 0.392092] acpiphp: Slot [14] registered [ 0.394089] acpiphp: Slot [15] registered [ 0.395084] acpiphp: Slot [16] registered [ 0.396113] acpiphp: Slot [17] registered [ 0.398179] acpiphp: Slot [18] registered [ 0.399082] acpiphp: Slot [19] registered [ 0.401086] acpiphp: Slot [20] registered [ 0.402000] acpiphp: Slot [21] registered [ 0.402000] acpiphp: Slot [22] registered [ 0.404102] acpiphp: Slot [23] registered [ 0.405087] acpiphp: Slot [24] registered [ 0.406131] acpiphp: Slot [25] registered [ 0.407085] acpiphp: Slot [26] registered [ 0.409074] acpiphp: Slot [27] registered [ 0.410073] acpiphp: Slot [28] registered [ 0.411086] acpiphp: Slot [29] registered [ 0.413091] acpiphp: Slot [30] registered [ 0.415117] acpiphp: Slot [31] registered [ 0.416069] PCI host bridge to bus 0000:00 [ 0.417018] pci_bus 0000:00: root bus resource [io 0x0000-0x0cf7 window] [ 0.420027] pci_bus 0000:00: root bus resource [io 0x0d00-0xffff window] [ 0.421000] pci_bus 0000:00: root bus resource [mem 0x000a0000-0x000bffff window] [ 0.422022] pci_bus 0000:00: root bus resource [mem 0xc0000000-0xfebfffff window] [ 0.423023] pci_bus 0000:00: root bus resource [mem 0x380000000000-0x38007fffffff window] [ 0.425119] pci_bus 0000:00: root bus resource [bus 00-ff] [ 0.426000] pci 0000:00:00.0: [8086:1237] type 00 class 0x060000 [ 0.426000] pci 0000:00:01.0: [8086:7000] type 00 class 0x060100 [ 0.426000] pci 0000:00:01.1: [8086:7010] type 00 class 0x010180 [ 0.431832] pci 0000:00:01.1: reg 0x20: [io 0xc320-0xc32f] [ 0.436061] pci 0000:00:01.1: legacy IDE quirk: reg 0x10: [io 0x01f0-0x01f7] [ 0.439020] pci 0000:00:01.1: legacy IDE quirk: reg 0x14: [io 0x03f6] [ 0.442023] pci 0000:00:01.1: legacy IDE quirk: reg 0x18: [io 0x0170-0x0177] [ 0.444023] pci 0000:00:01.1: legacy IDE quirk: reg 0x1c: [io 0x0376] [ 0.446686] pci 0000:00:01.3: [8086:7113] type 00 class 0x068000 [ 0.449732] pci 0000:00:01.3: quirk: [io 0x0600-0x063f] claimed by PIIX4 ACPI [ 0.452049] pci 0000:00:01.3: quirk: [io 0x0700-0x070f] claimed by PIIX4 SMB [ 0.455878] pci 0000:00:02.0: [1af4:1000] type 00 class 0x020000 [ 0.460016] pci 0000:00:02.0: reg 0x10: [io 0xc300-0xc31f] [ 0.470020] pci 0000:00:02.0: reg 0x20: [mem 0x380000000000-0x380000003fff 64bit pref] [ 0.475859] pci 0000:00:02.0: reg 0x30: [mem 0xfeb80000-0xfebbffff pref] [ 0.480938] pci 0000:00:05.0: [1af4:1001] type 00 class 0x010000 [ 0.494031] pci 0000:00:05.0: reg 0x10: [io 0xc000-0xc07f] [ 0.504023] pci 0000:00:05.0: reg 0x14: [mem 0xfebc0000-0xfebc0fff] [ 0.523024] pci 0000:00:05.0: reg 0x20: [mem 0x380000004000-0x380000007fff 64bit pref] [ 0.532767] pci 0000:00:06.0: [1af4:1001] type 00 class 0x010000 [ 0.538031] pci 0000:00:06.0: reg 0x10: [io 0xc080-0xc0ff] [ 0.543017] pci 0000:00:06.0: reg 0x14: [mem 0xfebc1000-0xfebc1fff] [ 0.552000] pci 0000:00:06.0: reg 0x20: [mem 0x380000008000-0x38000000bfff 64bit pref] [ 0.556000] pci 0000:00:07.0: [1af4:1001] type 00 class 0x010000 [ 0.562017] pci 0000:00:07.0: reg 0x10: [io 0xc100-0xc17f] [ 0.568020] pci 0000:00:07.0: reg 0x14: [mem 0xfebc2000-0xfebc2fff] [ 0.586042] pci 0000:00:07.0: reg 0x20: [mem 0x38000000c000-0x38000000ffff 64bit pref] [ 0.599910] pci 0000:00:08.0: [1af4:1001] type 00 class 0x010000 [ 0.614019] pci 0000:00:08.0: reg 0x10: [io 0xc180-0xc1ff] [ 0.625020] pci 0000:00:08.0: reg 0x14: [mem 0xfebc3000-0xfebc3fff] [ 0.646016] pci 0000:00:08.0: reg 0x20: [mem 0x380000010000-0x380000013fff 64bit pref] [ 0.655471] pci 0000:00:09.0: [1af4:1001] type 00 class 0x010000 [ 0.661016] pci 0000:00:09.0: reg 0x10: [io 0xc200-0xc27f] [ 0.666013] pci 0000:00:09.0: reg 0x14: [mem 0xfebc4000-0xfebc4fff] [ 0.676020] pci 0000:00:09.0: reg 0x20: [mem 0x380000014000-0x380000017fff 64bit pref] [ 0.684826] pci 0000:00:0a.0: [1af4:1001] type 00 class 0x010000 [ 0.691014] pci 0000:00:0a.0: reg 0x10: [io 0xc280-0xc2ff] [ 0.696020] pci 0000:00:0a.0: reg 0x14: [mem 0xfebc5000-0xfebc5fff] [ 0.709022] pci 0000:00:0a.0: reg 0x20: [mem 0x380000018000-0x38000001bfff 64bit pref] [ 0.724939] ACPI: PCI: Interrupt link LNKA configured for IRQ 10 [ 0.728409] ACPI: PCI: Interrupt link LNKB configured for IRQ 10 [ 0.730366] ACPI: PCI: Interrupt link LNKC configured for IRQ 11 [ 0.732373] ACPI: PCI: Interrupt link LNKD configured for IRQ 11 [ 0.735237] ACPI: PCI: Interrupt link LNKS configured for IRQ 9 [ 0.742185] iommu: Default domain type: Passthrough [ 0.744689] SCSI subsystem initialized [ 0.746196] ACPI: bus type USB registered [ 0.748145] usbcore: registered new interface driver usbfs [ 0.751473] usbcore: registered new interface driver hub [ 0.753130] usbcore: registered new device driver usb [ 0.755238] pps_core: LinuxPPS API ver. 1 registered [ 0.758014] pps_core: Software ver. 5.3.6 - Copyright 2005-2007 Rodolfo Giometti [ 0.760085] PTP clock support registered [ 0.763184] EDAC MC: Ver: 3.0.0 [ 0.765214] PCI: Using ACPI for IRQ routing [ 0.771209] NetLabel: Initializing [ 0.773016] NetLabel: domain hash size = 128 [ 0.775011] NetLabel: protocols = UNLABELED CIPSOv4 CALIPSO [ 0.778140] NetLabel: unlabeled traffic allowed by default [ 0.780351] vgaarb: loaded [ 0.782735] hpet0: at MMIO 0xfed00000, IRQs 2, 8, 0 [ 0.784009] hpet0: 3 comparators, 64-bit 100.000000 MHz counter [ 0.793147] clocksource: Switched to clocksource kvm-clock [ 0.922588] VFS: Disk quotas dquot_6.6.0 [ 0.924143] VFS: Dquot-cache hash table entries: 512 (order 0, 4096 bytes) [ 0.926189] *** VALIDATE ramfs *** [ 0.927551] *** VALIDATE hugetlbfs *** [ 0.929219] pnp: PnP ACPI init [ 0.931672] pnp: PnP ACPI: found 6 devices [ 0.947541] clocksource: acpi_pm: mask: 0xffffff max_cycles: 0xffffff, max_idle_ns: 2085701024 ns [ 0.950881] pci_bus 0000:00: resource 4 [io 0x0000-0x0cf7 window] [ 0.953222] pci_bus 0000:00: resource 5 [io 0x0d00-0xffff window] [ 0.955606] pci_bus 0000:00: resource 6 [mem 0x000a0000-0x000bffff window] [ 0.958198] pci_bus 0000:00: resource 7 [mem 0xc0000000-0xfebfffff window] [ 0.960831] pci_bus 0000:00: resource 8 [mem 0x380000000000-0x38007fffffff window] [ 0.963712] NET: Registered protocol family 2 [ 0.966235] IP idents hash table entries: 131072 (order: 8, 1048576 bytes, vmalloc) [ 0.971527] tcp_listen_portaddr_hash hash table entries: 4096 (order: 5, 163840 bytes, vmalloc) [ 0.977142] TCP established hash table entries: 65536 (order: 7, 524288 bytes, vmalloc) [ 0.982875] TCP bind hash table entries: 65536 (order: 9, 2097152 bytes, vmalloc) [ 0.986695] TCP: Hash tables configured (established 65536 bind 65536) [ 0.990242] MPTCP token hash table entries: 8192 (order: 6, 393216 bytes, vmalloc) [ 0.993517] UDP hash table entries: 4096 (order: 6, 393216 bytes, vmalloc) [ 0.996476] UDP-Lite hash table entries: 4096 (order: 6, 393216 bytes, vmalloc) [ 0.999735] NET: Registered protocol family 1 [ 1.002946] RPC: Registered named UNIX socket transport module. [ 1.004717] RPC: Registered udp transport module. [ 1.006109] RPC: Registered tcp transport module. [ 1.007866] RPC: Registered tcp NFSv4.1 backchannel transport module. [ 1.010686] NET: Registered protocol family 44 [ 1.012435] pci 0000:00:00.0: Limiting direct PCI/PCI transfers [ 1.014528] pci 0000:00:01.0: PIIX3: Enabling Passive Release [ 1.017099] pci 0000:00:01.0: Activating ISA DMA hang workarounds [ 1.019833] PCI: CLS 0 bytes, default 64 [ 1.021628] Unpacking initramfs... [ 2.782434] debug: unmapping init [mem 0xffff90617cc54000-0xffff90617ffbffff] [ 2.787850] PCI-DMA: Using software bounce buffering for IO (SWIOTLB) [ 2.792517] software IO TLB: mapped [mem 0x00000000a8000000-0x00000000ac000000] (64MB) [ 2.799128] clocksource: tsc: mask: 0xffffffffffffffff max_cycles: 0x229835b7123, max_idle_ns: 440795242976 ns [ 3.500608] Initialise system trusted keyrings [ 3.504138] Key type blacklist registered [ 3.508803] workingset: timestamp_bits=36 max_order=20 bucket_order=0 [ 3.520591] zbud: loaded [ 3.525186] *** VALIDATE nfs *** [ 3.526381] *** VALIDATE nfs4 *** [ 3.527889] pstore: using deflate compression [ 3.531636] Platform Keyring initialized [ 3.686204] NET: Registered protocol family 38 [ 3.688262] Key type asymmetric registered [ 3.689814] Asymmetric key parser 'x509' registered [ 3.692730] Block layer SCSI generic (bsg) driver version 0.4 loaded (major 247) [ 3.716205] io scheduler mq-deadline registered [ 3.718896] io scheduler kyber registered [ 3.720847] io scheduler bfq registered [ 3.726800] atomic64_test: passed for x86-64 platform with CX8 and with SSE [ 3.730456] shpchp: Standard Hot Plug PCI Controller Driver version: 0.4 [ 3.733437] input: Power Button as /devices/LNXSYSTM:00/LNXPWRBN:00/input/input0 [ 3.736458] ACPI: Power Button [PWRF] [ 3.834761] ACPI: \_SB_.LNKB: Enabled at IRQ 10 [ 3.939437] ACPI: \_SB_.LNKA: Enabled at IRQ 11 [ 4.180781] ACPI: \_SB_.LNKC: Enabled at IRQ 11 [ 4.342836] ACPI: \_SB_.LNKD: Enabled at IRQ 10 [ 4.595265] Serial: 8250/16550 driver, 4 ports, IRQ sharing enabled [ 4.638860] 00:03: ttyS1 at I/O 0x2f8 (irq = 3, base_baud = 115200) is a 16550A [ 4.679550] 00:04: ttyS0 at I/O 0x3f8 (irq = 4, base_baud = 115200) is a 16550A [ 4.687776] Non-volatile memory driver v1.3 [ 4.690230] Linux agpgart interface v0.103 [ 4.744494] virtio_blk virtio1: [vda] 133160 512-byte logical blocks (68.2 MB/65.0 MiB) [ 4.750348] vda: detected capacity change from 0 to 68177920 [ 4.806380] virtio_blk virtio2: [vdb] 2097152 512-byte logical blocks (1.07 GB/1.00 GiB) [ 4.813431] vdb: detected capacity change from 0 to 1073741824 [ 4.847134] virtio_blk virtio3: [vdc] 5120000 512-byte logical blocks (2.62 GB/2.44 GiB) [ 4.851877] vdc: detected capacity change from 0 to 2621440000 [ 4.878802] virtio_blk virtio4: [vdd] 5120000 512-byte logical blocks (2.62 GB/2.44 GiB) [ 4.883421] vdd: detected capacity change from 0 to 2621440000 [ 4.914799] virtio_blk virtio5: [vde] 8388608 512-byte logical blocks (4.29 GB/4.00 GiB) [ 4.920058] vde: detected capacity change from 0 to 4294967296 [ 4.944127] virtio_blk virtio6: [vdf] 8388608 512-byte logical blocks (4.29 GB/4.00 GiB) [ 4.946898] vdf: detected capacity change from 0 to 4294967296 [ 4.955232] libphy: Fixed MDIO Bus: probed [ 4.961878] usbcore: registered new interface driver usbserial_generic [ 4.965275] usbserial: USB Serial support registered for generic [ 4.967689] i8042: PNP: PS/2 Controller [PNP0303:KBD,PNP0f13:MOU] at 0x60,0x64 irq 1,12 [ 4.972280] serio: i8042 KBD port at 0x60,0x64 irq 1 [ 4.974116] serio: i8042 AUX port at 0x60,0x64 irq 12 [ 4.976429] mousedev: PS/2 mouse device common for all mice [ 4.979845] rtc_cmos 00:05: RTC can wake from S4 [ 4.982557] input: AT Translated Set 2 keyboard as /devices/platform/i8042/serio0/input/input1 [ 4.983047] rtc_cmos 00:05: registered as rtc0 [ 4.992514] rtc_cmos 00:05: alarms up to one day, y3k, 242 bytes nvram, hpet irqs [ 4.995043] intel_pstate: CPU model not supported [ 5.000412] input: VirtualPS/2 VMware VMMouse as /devices/platform/i8042/serio1/input/input4 [ 5.008167] hid: raw HID events driver (C) Jiri Kosina [ 5.011365] usbcore: registered new interface driver usbhid [ 5.014264] usbhid: USB HID core driver [ 5.015784] input: VirtualPS/2 VMware VMMouse as /devices/platform/i8042/serio1/input/input3 [ 5.020193] drop_monitor: Initializing network drop monitor service [ 5.022557] Initializing XFRM netlink socket [ 5.025162] NET: Registered protocol family 10 [ 5.029510] Segment Routing with IPv6 [ 5.030900] NET: Registered protocol family 17 [ 5.033893] mpls_gso: MPLS GSO support [ 5.039312] RAS: Correctable Errors collector initialized. [ 5.041352] AVX version of gcm_enc/dec engaged. [ 5.042901] AES CTR mode by8 optimization enabled [ 5.166959] sched_clock: Marking stable (5166605580, 0)->(6107958084, -941352504) [ 5.171775] registered taskstats version 1 [ 5.174085] Loading compiled-in X.509 certificates [ 5.175995] zswap: loaded using pool lzo/zbud [ 5.218513] Key type big_key registered [ 5.233756] Key type encrypted registered [ 5.235548] ima: No TPM chip found, activating TPM-bypass! [ 5.237690] ima: Allocated hash algorithm: sha1 [ 5.239593] ima: No architecture policies found [ 5.241462] evm: Initialising EVM extended attributes: [ 5.243561] evm: security.selinux [ 5.244842] evm: security.ima [ 5.246130] evm: security.capability [ 5.247618] evm: HMAC attrs: 0x1 [ 5.252406] rtc_cmos 00:05: setting system clock to 2025-09-24 18:03:11 UTC (1758736991) [ 5.263355] debug: unmapping init [mem 0xffffffffb7803000-0xffffffffb79fffff] [ 5.266441] debug: unmapping init [mem 0xffffffffb6582000-0xffffffffb6858fff] [ 5.277153] Write protecting the kernel read-only data: 28672k [ 5.282704] debug: unmapping init [mem 0xffffffffb4c03000-0xffffffffb4dfffff] [ 5.286015] debug: unmapping init [mem 0xffffffffb5514000-0xffffffffb55fffff] [ 5.329206] systemd[1]: systemd 239 (239-82.el8_10.5) running in system mode. (+PAM +AUDIT +SELINUX +IMA -APPARMOR +SMACK +SYSVINIT +UTMP +LIBCRYPTSETUP +GCRYPT +GNUTLS +ACL +XZ +LZ4 +SECCOMP +BLKID +ELFUTILS +KMOD +IDN2 -IDN +PCRE2 default-hierarchy=legacy) [ 5.339515] systemd[1]: Detected virtualization kvm. [ 5.340980] systemd[1]: Detected architecture x86-64. [ 5.342911] systemd[1]: Running in initial RAM disk. Welcome to Rocky Linux 8.10 (Green Obsidian) dracut-049-233.git20240115.el8 (Initramfs)! [ 5.376131] systemd[1]: No hostname configured. [ 5.377583] systemd[1]: Set hostname to . [ 5.379377] random: systemd: uninitialized urandom read (16 bytes read) [ 5.382073] systemd[1]: Initializing machine ID from random generator. [ 5.409094] hrtimer: interrupt took 13086845 ns [ 5.497526] random: ln: uninitialized urandom read (6 bytes read) [ 5.644395] random: systemd: uninitialized urandom read (16 bytes read) [ 5.648272] systemd[1]: Listening on Journal Socket. [ OK ] Listening on Journal Socket. [ 5.658122] systemd[1]: Reached target Slices. [ OK ] Reached target Slices. [ 5.669653] systemd[1]: Starting Create list of required static device nodes for the current kernel... Starting Create list of required st…ce nodes for the current kernel... [ OK ] Started Memstrack Anylazing Service. [ OK ] Reached target Initrd Root Device. Starting Setup Virtual Console... [ OK ] Listening on udev Kernel Socket. [ OK ] Reached target Timers. [ OK ] Listening on Journal Socket (/dev/log). Starting Journal Service... Starting Apply Kernel Variables... [ OK ] Reached target Local File Systems. Starting Create Volatile Files and Directories... [ OK ] Reached target Swap. [ OK ] Started Dispatch Password Requests to Console Directory Watch. [ OK ] Reached target Local Encrypted Volumes. [ OK ] Reached target Paths. [ OK ] Listening on udev Control Socket. [ OK ] Reached target Sockets. [ OK ] Started Create list of required sta…vice nodes for the current kernel. [ OK ] Started Setup Virtual Console. [ OK ] Started Apply Kernel Variables. [ OK ] Started Create Volatile Files and Directories. Starting dracut cmdline hook... Starting Create Static Device Nodes in /dev... [ OK ] Started Create Static Device Nodes in /dev. [ OK ] Started Journal Service. [ OK ] Started dracut cmdline hook. Starting dracut pre-udev hook... [ 7.038914] device-mapper: uevent: version 1.0.3 [ 7.041585] device-mapper: ioctl: 4.46.0-ioctl (2022-02-22) initialised: dm-devel@redhat.com [ OK ] Started dracut pre-udev hook. Starting udev Kernel Device Manager... [ OK ] Started udev Kernel Device Manager. Starting dracut pre-trigger hook... [ OK ] Started dracut pre-trigger hook. Starting udev Coldplug all Devices... Mounting Kernel Configuration File System... [ OK ] Mounted Kernel Configuration File System. [ OK ] Started udev Coldplug all Devices. Starting dracut initqueue hook... [ OK ] Reached target System Initialization. [ OK ] Reached target Basic System. [ OK ] Started Hardware RNG Entropy Gatherer Daemon. [ 8.330534] virtio_net virtio0 ens2: renamed from eth0 [ 8.504913] random: fast init done [ 8.741645] scsi host0: ata_piix [ 8.779066] scsi host1: ata_piix [ 8.786947] ata1: PATA max MWDMA2 cmd 0x1f0 ctl 0x3f6 bmdma 0xc320 irq 14 [ 8.796474] ata2: PATA max MWDMA2 cmd 0x170 ctl 0x376 bmdma 0xc328 irq 15 [ 13.758708] random: crng init done [ 13.764193] random: 7 urandom warning(s) missed due to ratelimiting [ 16.699227] dracut-initqueue[591]: RTNETLINK answers: File exists Starting nbd nbd0... [ OK ] Started nbd nbd0. [ OK ] Started dracut initqueue hook. [ OK ] Reached target Remote File Systems (Pre). [ OK ] Reached target Remote File Systems. Mounting /sysroot... [ 18.136972] EXT4-fs (nbd0): mounted filesystem with ordered data mode. Opts: (null) [ OK ] Mounted /sysroot. [ OK ] Reached target Initrd Root File System. Starting Reload Configuration from the Real Root... [ OK ] Started Reload Configuration from the Real Root. [ OK ] Reached target Initrd File Systems. [ OK ] Reached target Initrd Default Target. Starting dracut pre-pivot and cleanup hook... [ OK ] Started dracut pre-pivot and cleanup hook. Starting Cleaning Up and Shutting Down Daemons... Stopping Hardware RNG Entropy Gatherer Daemon... [ OK ] Stopped target Timers. [ OK ] Stopped dracut pre-pivot and cleanup hook. [ OK ] Stopped target Initrd Default Target. [ OK ] Stopped target Remote File Systems. [ OK ] Stopped target Remote File Systems (Pre). [ OK ] Stopped dracut initqueue hook. [ OK ] Stopped target Initrd Root Device. [ OK ] Stopped Hardware RNG Entropy Gatherer Daemon. [ OK ] Stopped target Basic System. [ OK ] Stopped target Paths. [ OK ] Stopped target Slices. [ OK ] Stopped target Sockets. [ OK ] Stopped target System Initialization. [ OK ] Stopped udev Coldplug all Devices. [ OK ] Stopped dracut pre-trigger hook. [ OK ] Stopped target Swap. Stopping udev Kernel Device Manager... [ OK ] Stopped Apply Kernel Variables. [ OK ] Stopped Create Volatile Files and Directories. [ OK ] Stopped target Local Encrypted Volumes. [ OK ] Stopped Dispatch Password Requests to Console Directory Watch. [ OK ] Stopped target Local File Systems. [ OK ] Stopped udev Kernel Device Manager. [ OK ] Stopped Create Static Device Nodes in /dev. [ OK ] Stopped Create list of required sta…vice nodes for the current kernel. [ OK ] Stopped dracut pre-udev hook. [ OK ] Stopped dracut cmdline hook. [ OK ] Closed udev Kernel Socket. [ OK ] Closed udev Control Socket. Starting Cleanup udevd DB... [ OK ] Started Cleaning Up and Shutting Down Daemons. [ OK ] Started Cleanup udevd DB. [ OK ] Reached target Switch Root. Starting Switch Root... [ 19.844455] printk: systemd: 26 output lines suppressed due to ratelimiting [ 20.266544] SELinux: Disabled at runtime. [ 20.372654] systemd[1]: systemd 239 (239-82.el8_10.5) running in system mode. (+PAM +AUDIT +SELINUX +IMA -APPARMOR +SMACK +SYSVINIT +UTMP +LIBCRYPTSETUP +GCRYPT +GNUTLS +ACL +XZ +LZ4 +SECCOMP +BLKID +ELFUTILS +KMOD +IDN2 -IDN +PCRE2 default-hierarchy=legacy) [ 20.389059] systemd[1]: Detected virtualization kvm. [ 20.392296] systemd[1]: Detected architecture x86-64. Welcome to Rocky Linux 8.10 (Green Obsidian)! [ 21.322220] systemd[1]: initrd-switch-root.service: Succeeded. [ 21.327895] systemd[1]: Stopped Switch Root. [ OK ] Stopped Switch Root. [ 21.355400] systemd[1]: systemd-journald.service: Service has no hold-off time (RestartSec=0), scheduling restart. [ 21.390549] systemd[1]: systemd-journald.service: Scheduled restart job, restart counter is at 1. [ 21.394491] systemd[1]: Stopped Journal Service. [ OK ] Stopped Journal Service. [ 21.405870] systemd[1]: Starting Journal Service... Starting Journal Service... [ 21.425352] systemd[1]: Starting Remount Root and Kernel File Systems... Starting Remount Root and Kernel File Systems... [ OK ] Reached target rpc_pipefs.target. [ OK ] Listening on initctl Compatibility Named Pipe. [ OK ] Stopped target Switch Root. Starting Create list of required st…ce nodes for the current kernel... Mounting Huge Pages File System... [ OK ] Listening on RPCbind Server Activation Socket. Mounting POSIX Message Queue File System... [ OK ] Stopped target Initrd File Systems. [ OK ] Created slice system-serial\x2dgetty.slice. [ OK ] Reached target RPC Port Mapper. [FAILED] Failed to set up automount Arbitrar…rmats File System Automount Point. See 'systemctl status proc-sys-fs-binfmt_misc.automount' for details. [ OK ] Created slice User and Session Slice. [ OK ] Created slice system-getty.slice. Mounting Kernel Debug File System... [ OK ] Stopped target Initrd Root File System. [ OK ] Listening on Process Core Dump Socket. [ OK ] Started Forward Password Requests to Wall Directory Watch. [ OK ] Listening on udev Control Socket. [ OK ] Listening on udev Kernel Socket. Starting udev Coldplug all Devices... [ OK ] Reached target Slices. [ OK ] Created slice system-sshd\x2dkeygen.slice. [ OK ] Started Dispatch Password Requests to Console Directory Watch. [ OK ] Reached target Local Encrypted Volumes. [ OK ] Reached target Paths. Activating swap /dev/disk/by-label/SWAP... Starting Apply Kernel Variables... [ 21.984263] Adding 1048572k swap on /dev/vdb. Priority:-2 extents:1 across:1048572k FS [ OK ] Started Journal Service. [FAILED] Failed to start Remount Root and Kernel File Systems. See 'systemctl status systemd-remount-fs.service' for details. [ OK ] Started Create list of required sta…vice nodes for the current kernel. [ OK ] Mounted Huge Pages File System. [ OK ] Mounted POSIX Message Queue File System. [ OK ] Mounted Kernel Debug File System. [ OK ] Activated swap /dev/disk/by-label/SWAP. [ OK ] Started Apply Kernel Variables. [ OK ] Reached target Swap. Starting Configure read-only root support... Starting Create Static Device Nodes in /dev... Starting Flush Journal to Persistent Storage... [ OK ] Started Flush Journal to Persistent Storage. [ OK ] Started udev Coldplug all Devices. [ OK ] Started Create Static Device Nodes in /dev. Starting udev Kernel Device Manager... [ OK ] Reached target Local File Systems (Pre). Mounting /home/green/git/lustre-release... Mounting /mnt... [ OK ] Mounted /mnt. [ 22.942103] squashfs: version 4.0 (2009/01/31) Phillip Lougher [ OK ] Mounted /home/green/git/lustre-release. [ OK ] Started udev Kernel Device Manager. [ 23.899191] piix4_smbus 0000:00:01.3: SMBus Host Controller at 0x700, revision 0 [ 23.950884] input: PC Speaker as /devices/platform/pcspkr/input/input5 [ 24.747407] RAPL PMU: API unit is 2^-32 Joules, 0 fixed counters, 10737418240 ms ovfl timer [ 24.990906] EDAC sbridge: Ver: 1.1.2 [* ] A start job is running for Configur…-only root support (7s / no limit) [** ] A start job is running for Configur…-only root support (7s / no limit) [*** ] A start job is running for Configur…-only root support (8s / no limit)[ 30.118723] Key type dns_resolver registered [ *** ] A start job is running for Configur…-only root support (8s / no limit)[ 30.550288] NFS: Registering the id_resolver key type [ 30.552522] Key type id_resolver registered [ 30.555398] Key type id_legacy registered [ *** ] A start job is running for Configur…-only root support (9s / no limit) [ ***] A start job is running for Configur…-only root support (9s / no limit) [ OK ] Started Configure read-only root support. [ OK ] Reached target Local File Systems. Starting Mark the need to relabel after reboot... Starting Rebuild Dynamic Linker Cache... Starting Create Volatile Files and Directories... Starting Load/Save Random Seed... [ OK ] Started Mark the need to relabel after reboot. [ OK ] Started Create Volatile Files and Directories. [ OK ] Started Load/Save Random Seed. Starting Update UTMP about System Boot/Shutdown... Starting RPC Bind... [ OK ] Started Update UTMP about System Boot/Shutdown. [ OK ] Started RPC Bind. [ OK ] Started Rebuild Dynamic Linker Cache. Starting Update is Completed... [ OK ] Started Update is Completed. [ OK ] Reached target System Initialization. [ OK ] Started dnf makecache --timer. [ OK ] Started Daily Cleanup of Temporary Directories. [ OK ] Started daily update of the root trust anchor for DNSSEC. [ OK ] Reached target Timers. [ OK ] Listening on D-Bus System Message Bus Socket. [ OK ] Reached target Sockets. [ OK ] Reached target Basic System. Starting Login Service... [ OK ] Started Hardware RNG Entropy Gatherer Daemon. Starting Restore /run/initramfs on shutdown... [ OK ] Reached target sshd-keygen.target. [ OK ] Started irqbalance daemon. [ OK ] Started D-Bus System Message Bus. Starting Network Manager... [ OK ] Started Restore /run/initramfs on shutdown. [ OK ] Started Network Manager. [ OK ] Reached target Network. Starting GSSAPI Proxy Daemon... Starting OpenSSH server daemon... Starting Dynamic System Tuning Daemon... Starting Network Manager Wait Online... [ OK ] Started Login Service. [ OK ] Started GSSAPI Proxy Daemon. Starting Hostname Service... [ OK ] Reached target NFS client services. [ OK ] Reached target Remote File Systems (Pre). [ OK ] Reached target Remote File Systems. Starting Permit User Sessions... [ OK ] Started OpenSSH server daemon. [ OK ] Started Permit User Sessions. [ OK ] Started Getty on tty1. [ OK ] Started Serial Getty on ttyS1. [ OK ] Started Serial Getty on ttyS0. [ OK ] Reached target Login Prompts. [ OK ] Started Command Scheduler. [ OK ] Started Hostname Service. Starting Network Manager Script Dispatcher Service... [ OK ] Started Network Manager Script Dispatcher Service. [ OK ] Started Network Manager Wait Online. [ OK ] Reached target Network is Online. Starting System Logging Service... Starting Crash recovery kernel arming... Starting Notify NFS peers of a restart... [ OK ] Started Notify NFS peers of a restart. [ OK ] Started System Logging Service. Rocky Linux 8.10 (Green Obsidian) Kernel 4.18.0rh8.10-debug on an x86_64 oleg158-server login: [ 73.899257] libcfs: loading out-of-tree module taints kernel. [ 73.930997] Key type ._llcrypt registered [ 73.932738] Key type .llcrypt registered [ 74.002311] Lustre: DEBUG MARKER: oleg158-server.virtnet: executing set_hostid [ 85.515174] Lustre: DEBUG MARKER: oleg158-server.virtnet: executing load_modules_local [ 86.506970] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 86.520632] alg: No test for adler32 (adler32-zlib) [ 87.691553] Lustre: Lustre: Build Version: 2.16.58_53_g2e3f3ed [ 88.251321] LNet: Added LNI 192.168.201.158@tcp [8/256/0/180] [ 89.928394] Key type lgssc registered [ 91.022188] Lustre: Echo OBD driver; http://www.lustre.org/ [ 101.382555] ZFS: Loaded module v2.3.2-1, ZFS pool version 5000, ZFS filesystem version 5 [ 103.099464] LDISKFS-fs (vdc): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 109.188802] LDISKFS-fs (vdd): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 114.013109] LDISKFS-fs (vde): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 118.918273] LDISKFS-fs (vdf): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 128.198208] Lustre: DEBUG MARKER: oleg158-server.virtnet: executing load_modules_local [ 136.531816] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 136.571663] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 136.594902] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 137.783379] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 137.814021] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 137.900398] Lustre: lustre-MDT0000: new disk, initializing [ 137.986238] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 138.002738] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 140.446499] Lustre: DEBUG MARKER: oleg158-server.virtnet: executing set_default_debug vfstrace rpctrace dlmtrace neterror ha config ioctl super lfsck all [ 148.810620] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 148.867905] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 148.917628] Lustre: 6490:0:(mgs_llog.c:1345:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 148.939613] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [ 148.943074] Lustre: Skipped 1 previous similar message [ 148.993230] Lustre: lustre-MDT0001: new disk, initializing [ 149.038117] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 149.058955] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [ 149.070573] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 151.357603] Lustre: DEBUG MARKER: oleg158-server.virtnet: executing set_default_debug vfstrace rpctrace dlmtrace neterror ha config ioctl super lfsck all [ 154.599521] Lustre: Modifying parameter general.debug_raw_pointers=Y in log params [ 160.144501] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 160.183853] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 160.338158] Lustre: lustre-OST0000: new disk, initializing [ 160.340969] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 160.379991] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 163.150303] Lustre: DEBUG MARKER: oleg158-server.virtnet: executing set_default_debug vfstrace rpctrace dlmtrace neterror ha config ioctl super lfsck all [ 164.044333] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [ 164.048886] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 164.094559] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 172.345742] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 172.442142] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 172.515170] Lustre: lustre-OST0001: new disk, initializing [ 172.518838] Lustre: srv-lustre-OST0001: No data found on store. Initialize space. [ 172.561923] Lustre: lustre-OST0001: Imperative Recovery not enabled, recovery window 60-180 [ 176.150555] Lustre: DEBUG MARKER: oleg158-server.virtnet: executing set_default_debug vfstrace rpctrace dlmtrace neterror ha config ioctl super lfsck all [ 177.662216] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x00000002c0000400-0x0000000300000400]:1:ost [ 177.666629] Lustre: cli-lustre-OST0001-super: Allocated super-sequence [0x00000002c0000400-0x0000000300000400]:1:ost] [ 177.693277] Lustre: lustre-OST0001-osc-MDT0000: update sequence from 0x100010000 to 0x2c0000400 [ 185.313838] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 192.708631] Lustre: Setting parameter general.lod.*.mdt_hash=crush in log params [ 200.820416] Lustre: DEBUG MARKER: oleg158-server.virtnet: executing check_logdir /tmp/testlogs/ [ 203.353512] Lustre: DEBUG MARKER: oleg158-server.virtnet: executing yml_node [ 205.811680] Lustre: DEBUG MARKER: Client: 2.16.58.53 [ 207.202689] Lustre: DEBUG MARKER: MDS: 2.16.58.53 [ 208.721187] Lustre: DEBUG MARKER: OSS: 2.16.58.53 [ 209.649577] Lustre: DEBUG MARKER: -----============= acceptance-small: conf-sanity ============----- Wed Sep 24 14:06:36 EDT 2025 [ 221.064883] Lustre: DEBUG MARKER: excepting tests: 102 106 115 32newtarball [ 222.021668] Lustre: DEBUG MARKER: skipping tests SLOW=no: 45 69 106 111 114 [ 236.001628] Lustre: lustre-MDT0000-lwp-OST0000: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 236.004763] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 236.010476] Lustre: Skipped 1 previous similar message [ 241.121694] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 241.124724] Lustre: Skipped 6 previous similar messages [ 241.420151] LustreError: 12189:0:(obd_class.h:479:obd_check_dev()) Device 24 not setup [ 241.518045] Lustre: server umount lustre-MDT0000 complete [ 244.140852] LustreError: 9456:0:(ldlm_lockd.c:2607:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1758737230 with bad export cookie 8933148830719295062 [ 244.141382] LustreError: MGC192.168.201.158@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 244.148419] LustreError: 9456:0:(ldlm_lockd.c:2607:ldlm_cancel_handler()) Skipped 3 previous similar messages [ 246.247891] Lustre: lustre-MDT0001-lwp-OST0000: Connection to lustre-MDT0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 246.252352] LustreError: 9191:0:(ldlm_lib.c:1132:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 246.255477] Lustre: lustre-MDT0001: Not available for connect from 0@lo (stopping) [ 246.259215] Lustre: Skipped 3 previous similar messages [ 246.283947] LustreError: 9191:0:(ldlm_lib.c:1132:target_handle_connect()) Skipped 3 previous similar messages [ 249.251397] LustreError: 6497:0:(ldlm_lib.c:1132:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 249.255984] Lustre: lustre-MDT0001: Not available for connect from 0@lo (stopping) [ 249.281330] Lustre: Skipped 2 previous similar messages [ 250.317437] LustreError: 12391:0:(obd_class.h:479:obd_check_dev()) Device 28 not setup [ 250.327514] LustreError: 12391:0:(obd_class.h:479:obd_check_dev()) Skipped 7 previous similar messages [ 250.450559] Lustre: server umount lustre-MDT0001 complete [ 252.829889] LustreError: 12593:0:(obd_class.h:479:obd_check_dev()) Device 19 not setup [ 252.834253] LustreError: 12593:0:(obd_class.h:479:obd_check_dev()) Skipped 7 previous similar messages [ 252.869926] Lustre: server umount lustre-OST0000 complete [ 255.291711] LustreError: 12793:0:(obd_class.h:479:obd_check_dev()) Device 2 not setup [ 255.301686] LustreError: 12793:0:(obd_class.h:479:obd_check_dev()) Skipped 3 previous similar messages [ 255.423657] Lustre: server umount lustre-OST0001 complete [ 259.054594] Lustre: DEBUG MARKER: oleg158-server.virtnet: executing set_hostid [ 264.578916] Lustre: DEBUG MARKER: oleg158-server.virtnet: executing load_modules_local [ 270.806784] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 274.431460] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 277.995839] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 281.651299] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 285.751529] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 285.794472] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 285.952294] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 285.983346] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 286.040767] Lustre: lustre-MDT0000: new disk, initializing [ 286.115037] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 286.136107] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 288.846857] Lustre: DEBUG MARKER: oleg158-server.virtnet: executing set_default_debug -1 all [ 295.923526] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 295.983313] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 296.046802] Lustre: 14976:0:(mgs_llog.c:1345:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 296.085520] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [ 296.088370] Lustre: Skipped 1 previous similar message [ 296.145314] Lustre: lustre-MDT0001: new disk, initializing [ 296.214244] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 296.245514] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [ 296.256150] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 298.416529] Lustre: DEBUG MARKER: oleg158-server.virtnet: executing set_default_debug -1 all [ 303.296802] Lustre: DEBUG MARKER: oleg158-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 305.646290] Lustre: DEBUG MARKER: oleg158-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 309.973639] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 310.036882] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 311.362940] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 311.551619] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 314.142637] Lustre: DEBUG MARKER: oleg158-server.virtnet: executing set_default_debug -1 all [ 325.160695] Lustre: DEBUG MARKER: oleg158-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 331.941037] Lustre: DEBUG MARKER: oleg158-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 [ 332.316918] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [ 337.944508] Lustre: DEBUG MARKER: oleg158-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid 50 [ 338.249959] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [ 342.502461] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 342.506179] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 342.518943] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 342.547055] Lustre: Skipped 1 previous similar message [ 342.552918] Lustre: Skipped 1 previous similar message [ 352.745572] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 352.758269] Lustre: Skipped 3 previous similar messages [ 355.296298] Lustre: lustre-OST0000 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 355.357098] LustreError: 17915:0:(obd_class.h:479:obd_check_dev()) Device 19 not setup [ 355.405071] LustreError: 17915:0:(obd_class.h:479:obd_check_dev()) Skipped 5 previous similar messages [ 355.723507] Lustre: server umount lustre-OST0000 complete [ 362.977230] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 362.978185] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 363.013301] Lustre: Skipped 1 previous similar message [ 363.026853] LustreError: Skipped 1 previous similar message [ 366.870727] LustreError: 18118:0:(obd_class.h:479:obd_check_dev()) Device 22 not setup [ 366.878512] LustreError: 18118:0:(obd_class.h:479:obd_check_dev()) Skipped 3 previous similar messages [ 367.094867] Lustre: server umount lustre-MDT0000 complete [ 368.116379] LustreError: 15948:0:(ldlm_lib.c:1132:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 368.169401] LustreError: 15948:0:(ldlm_lib.c:1132:target_handle_connect()) Skipped 2 previous similar messages [ 372.054850] LustreError: 15765:0:(ldlm_lockd.c:2607:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1758737358 with bad export cookie 8933148830719296595 [ 372.057644] LustreError: MGC192.168.201.158@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 372.122201] LustreError: 15765:0:(ldlm_lockd.c:2607:ldlm_cancel_handler()) Skipped 2 previous similar messages [ 372.781845] Lustre: server umount lustre-MDT0001 complete [ 380.066930] Lustre: DEBUG MARKER: == conf-sanity test 76a: set permanent params with lctl across mounts ========================================================== 14:09:25 (1758737365) [ 387.598685] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 387.948256] LustreError: 18849:0:(ldlm_lib.c:1132:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 388.045561] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 388.050890] Lustre: Skipped 1 previous similar message [ 391.798881] Lustre: DEBUG MARKER: oleg158-server.virtnet: executing set_default_debug -1 all [ 393.186892] LustreError: 18848:0:(ldlm_lib.c:1132:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 393.200822] LustreError: 18848:0:(ldlm_lib.c:1132:target_handle_connect()) Skipped 1 previous similar message [ 399.028826] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 401.993355] Lustre: DEBUG MARKER: oleg158-server.virtnet: executing set_default_debug -1 all [ 406.532131] Lustre: DEBUG MARKER: oleg158-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 409.453499] Lustre: DEBUG MARKER: oleg158-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 414.812515] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 419.579087] Lustre: DEBUG MARKER: oleg158-server.virtnet: executing set_default_debug -1 all [ 423.369329] Lustre: DEBUG MARKER: oleg158-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 436.776132] Lustre: Modifying parameter general.osc.*.max_dirty_mb=452 in log params [ 443.877296] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 443.883538] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 443.886299] Lustre: Skipped 4 previous similar messages [ 447.773946] LustreError: 21169:0:(obd_class.h:479:obd_check_dev()) Device 9 not setup [ 447.784043] LustreError: 21169:0:(obd_class.h:479:obd_check_dev()) Skipped 13 previous similar messages [ 447.872882] Lustre: server umount lustre-MDT0000 complete [ 448.997577] LustreError: 20853:0:(ldlm_lib.c:1132:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 449.018614] LustreError: 20853:0:(ldlm_lib.c:1132:target_handle_connect()) Skipped 3 previous similar messages [ 450.784122] LustreError: 18829:0:(ldlm_lockd.c:2607:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1758737437 with bad export cookie 8933148830719297477 [ 450.788368] LustreError: MGC192.168.201.158@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 450.804059] LustreError: 18829:0:(ldlm_lockd.c:2607:ldlm_cancel_handler()) Skipped 1 previous similar message [ 451.141740] Lustre: server umount lustre-MDT0001 complete [ 464.761564] Lustre: server umount lustre-OST0000 complete [ 483.533396] Lustre: DEBUG MARKER: oleg158-server.virtnet: executing load_modules_local [ 492.418244] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 492.810941] LustreError: 22900:0:(ldlm_lib.c:1132:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 492.865104] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 492.873276] Lustre: Skipped 2 previous similar messages [ 496.421639] Lustre: DEBUG MARKER: oleg158-server.virtnet: executing set_default_debug -1 all [ 503.661710] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 507.586592] Lustre: DEBUG MARKER: oleg158-server.virtnet: executing set_default_debug -1 all [ 510.309601] Lustre: Setting parameter general.debug_raw_pointers=Y in log params [ 516.155573] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 521.536478] Lustre: DEBUG MARKER: oleg158-server.virtnet: executing set_default_debug -1 all [ 529.267604] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 529.332439] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 529.491561] Lustre: lustre-OST0001: new disk, initializing [ 529.493944] Lustre: Skipped 1 previous similar message [ 529.503532] Lustre: srv-lustre-OST0001: No data found on store. Initialize space. [ 529.513662] Lustre: Skipped 1 previous similar message [ 530.615792] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x00000002c0000400-0x0000000300000400]:1:ost [ 530.626715] Lustre: Skipped 1 previous similar message [ 530.632277] Lustre: cli-lustre-OST0001-super: Allocated super-sequence [0x00000002c0000400-0x0000000300000400]:1:ost] [ 530.676461] Lustre: lustre-OST0001-osc-MDT0000: update sequence from 0x100010000 to 0x2c0000401 [ 534.018085] Lustre: DEBUG MARKER: oleg158-server.virtnet: executing set_default_debug -1 all [ 542.587351] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 545.924458] Lustre: Setting parameter general.lod.*.mdt_hash=crush in log params [ 562.658402] Lustre: lustre-MDT0000-lwp-OST0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 562.661962] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 562.683609] Lustre: Skipped 5 previous similar messages [ 562.690199] Lustre: Skipped 2 previous similar messages [ 568.132028] LustreError: 26379:0:(obd_class.h:479:obd_check_dev()) Device 28 not setup [ 568.141699] LustreError: 26379:0:(obd_class.h:479:obd_check_dev()) Skipped 17 previous similar messages [ 568.293479] Lustre: server umount lustre-MDT0000 complete [ 572.064390] LustreError: 22881:0:(ldlm_lockd.c:2607:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1758737558 with bad export cookie 8933148830719298324 [ 572.068603] LustreError: MGC192.168.201.158@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 572.608982] Lustre: server umount lustre-MDT0001 complete [ 586.754820] Lustre: server umount lustre-OST0000 complete [ 589.286107] Lustre: 3633:0:(client.c:2464:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1758737559/real 1758737559] req@ffff9060c4e3c380 x1844169486328448/t0(0) o400->lustre-MDT0001-lwp-OST0001@0@lo:12/10 lens 224/224 e 0 to 1 dl 1758737575 ref 1 fl Rpc:XNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 [ 589.345754] Lustre: lustre-MDT0001-lwp-OST0001: Connection to lustre-MDT0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 591.339406] Lustre: server umount lustre-OST0001 complete [ 609.872074] Lustre: DEBUG MARKER: oleg158-server.virtnet: executing load_modules_local [ 621.241930] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 621.631289] LustreError: 28157:0:(ldlm_lib.c:1132:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 621.654417] LustreError: 28157:0:(ldlm_lib.c:1132:target_handle_connect()) Skipped 2 previous similar messages [ 621.736224] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 621.749778] Lustre: Skipped 3 previous similar messages [ 626.643691] Lustre: DEBUG MARKER: oleg158-server.virtnet: executing set_default_debug -1 all [ 636.437715] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 641.112708] Lustre: DEBUG MARKER: oleg158-server.virtnet: executing set_default_debug -1 all [ 644.299514] Lustre: 29275:0:(mgs_llog.c:1345:mgs_modify_param()) MGS: modify general/debug_raw_pointers=Y (mode = 0) failed: rc = -17 [ 650.470397] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 656.562582] Lustre: DEBUG MARKER: oleg158-server.virtnet: executing set_default_debug -1 all [ 664.290083] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 670.624984] Lustre: DEBUG MARKER: oleg158-server.virtnet: executing set_default_debug -1 all [ 678.480686] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 682.286391] Lustre: 31128:0:(mgs_llog.c:1345:mgs_modify_param()) MGS: modify general/lod.*.mdt_hash=crush (mode = 0) failed: rc = -17 [ 697.856460] Lustre: lustre-MDT0000-lwp-OST0000: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 697.878317] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 697.908067] Lustre: Skipped 2 previous similar messages [ 697.934719] Lustre: Skipped 8 previous similar messages [ 699.830127] LustreError: 31332:0:(obd_class.h:479:obd_check_dev()) Device 10 not setup [ 699.871728] LustreError: 31332:0:(obd_class.h:479:obd_check_dev()) Skipped 25 previous similar messages [ 700.097491] Lustre: server umount lustre-MDT0000 complete [ 702.965485] LustreError: 28151:0:(ldlm_lib.c:1132:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 702.980645] LustreError: 28151:0:(ldlm_lib.c:1132:target_handle_connect()) Skipped 7 previous similar messages [ 704.247494] LustreError: 28138:0:(ldlm_lockd.c:2607:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1758737690 with bad export cookie 8933148830719299619 [ 704.266276] LustreError: MGC192.168.201.158@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 704.700817] Lustre: server umount lustre-MDT0001 complete [ 719.464871] Lustre: server umount lustre-OST0000 complete [ 733.393859] Lustre: server umount lustre-OST0001 complete [ 743.894681] Lustre: DEBUG MARKER: == conf-sanity test 76b: verify params log setup correctly ========================================================== 14:15:29 (1758737729) [ 771.554996] Lustre: DEBUG MARKER: oleg158-server.virtnet: executing load_modules_local [ 782.115404] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 782.683427] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 782.694975] Lustre: Skipped 3 previous similar messages [ 786.628339] Lustre: DEBUG MARKER: oleg158-server.virtnet: executing set_default_debug -1 all [ 795.499125] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 800.184115] Lustre: DEBUG MARKER: oleg158-server.virtnet: executing set_default_debug -1 all [ 803.166704] Lustre: 35033:0:(mgs_llog.c:1345:mgs_modify_param()) MGS: modify general/debug_raw_pointers=Y (mode = 0) failed: rc = -17 [ 809.530883] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 815.694632] Lustre: DEBUG MARKER: oleg158-server.virtnet: executing set_default_debug -1 all [ 823.955600] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 829.824810] Lustre: DEBUG MARKER: oleg158-server.virtnet: executing set_default_debug -1 all [ 837.194365] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 840.771303] Lustre: 36890:0:(mgs_llog.c:1345:mgs_modify_param()) MGS: modify general/lod.*.mdt_hash=crush (mode = 0) failed: rc = -17 [ 852.449303] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 852.452709] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 852.490153] Lustre: Skipped 1 previous similar message [ 852.502048] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 856.038718] LustreError: 37042:0:(obd_class.h:479:obd_check_dev()) Device 10 not setup [ 856.041864] LustreError: 37042:0:(obd_class.h:479:obd_check_dev()) Skipped 25 previous similar messages [ 856.366567] Lustre: server umount lustre-MDT0000 complete [ 858.597461] LustreError: 36657:0:(ldlm_lib.c:1132:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 858.624885] LustreError: 36657:0:(ldlm_lib.c:1132:target_handle_connect()) Skipped 7 previous similar messages [ 860.508164] LustreError: 33899:0:(ldlm_lockd.c:2607:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1758737846 with bad export cookie 8933148830719300725 [ 860.512061] LustreError: MGC192.168.201.158@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 860.521744] LustreError: 33899:0:(ldlm_lockd.c:2607:ldlm_cancel_handler()) Skipped 8 previous similar messages [ 860.972729] Lustre: server umount lustre-MDT0001 complete [ 875.200502] Lustre: server umount lustre-OST0000 complete [ 888.977575] Lustre: server umount lustre-OST0001 complete [ 896.971639] Lustre: DEBUG MARKER: == conf-sanity test 76c: verify changelog_mask is applied with lctl set_param -P ========================================================== 14:18:03 (1758737883) [ 914.378602] Lustre: DEBUG MARKER: oleg158-server.virtnet: executing load_modules_local [ 925.927771] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 930.800211] Lustre: DEBUG MARKER: oleg158-server.virtnet: executing set_default_debug -1 all [ 941.221146] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 947.678882] Lustre: DEBUG MARKER: oleg158-server.virtnet: executing set_default_debug -1 all [ 951.832371] Lustre: 40136:0:(mgs_llog.c:1345:mgs_modify_param()) MGS: modify general/debug_raw_pointers=Y (mode = 0) failed: rc = -17 [ 960.383767] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 967.323285] Lustre: DEBUG MARKER: oleg158-server.virtnet: executing set_default_debug -1 all [ 976.997895] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 983.674757] Lustre: DEBUG MARKER: oleg158-server.virtnet: executing set_default_debug -1 all [ 992.229709] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 995.669616] Lustre: 41992:0:(mgs_llog.c:1345:mgs_modify_param()) MGS: modify general/lod.*.mdt_hash=crush (mode = 0) failed: rc = -17 [ 1001.678364] Lustre: Setting parameter general.mdd.*.changelog_mask=-CLOSE in log params [ 1001.687358] Lustre: Skipped 1 previous similar message [ 1008.100156] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 1008.128750] Lustre: Skipped 6 previous similar messages [ 1011.129500] Lustre: server umount lustre-MDT0000 complete [ 1015.348164] LustreError: 39000:0:(ldlm_lockd.c:2607:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1758738001 with bad export cookie 8933148830719301831 [ 1015.361537] LustreError: MGC192.168.201.158@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 1021.823886] Lustre: server umount lustre-MDT0001 complete [ 1031.744261] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1040.395573] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 1040.414637] Lustre: Skipped 7 previous similar messages [ 1044.728941] Lustre: DEBUG MARKER: oleg158-server.virtnet: executing set_default_debug -1 all [ 1050.606117] Lustre: lustre-MDT0000-lwp-OST0000: Connection restored to 0@lo (at 0@lo) [ 1053.795920] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 1058.163284] Lustre: DEBUG MARKER: oleg158-server.virtnet: executing set_default_debug -1 all [ 1059.833495] Lustre: lustre-MDT0001-lwp-OST0000: Connection restored to 0@lo (at 0@lo) [ 1059.850712] Lustre: Skipped 1 previous similar message [ 1064.056777] Lustre: DEBUG MARKER: oleg158-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 1066.094989] Lustre: DEBUG MARKER: mdc.lustre-MDT0000-mdc-*.mds_server_uuid in FULL state after 0 sec [ 1070.327707] Lustre: DEBUG MARKER: oleg158-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 1072.663126] Lustre: DEBUG MARKER: mdc.lustre-MDT0001-mdc-*.mds_server_uuid in FULL state after 0 sec [ 1079.777309] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 1084.407930] Lustre: server umount lustre-MDT0000 complete [ 1088.658074] LustreError: 39000:0:(ldlm_lockd.c:2607:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1758738074 with bad export cookie 8933148830719302993 [ 1088.667623] LustreError: MGC192.168.201.158@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 1088.674418] LustreError: 39000:0:(ldlm_lockd.c:2607:ldlm_cancel_handler()) Skipped 8 previous similar messages [ 1089.149858] Lustre: server umount lustre-MDT0001 complete [ 1102.979097] Lustre: server umount lustre-OST0000 complete [ 1106.722114] Lustre: 3633:0:(client.c:2464:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1758738076/real 1758738076] req@ffff9061fe5ec000 x1844169486523776/t0(0) o400->lustre-MDT0001-lwp-OST0001@0@lo:12/10 lens 224/224 e 0 to 1 dl 1758738092 ref 1 fl Rpc:XNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 [ 1107.734565] Lustre: server umount lustre-OST0001 complete [ 1116.690625] Lustre: DEBUG MARKER: == conf-sanity test 76d: verify llite.*.xattr_cache can be set by 'lctl set_param -P' correctly ========================================================== 14:21:42 (1758738102) [ 1131.982687] Lustre: DEBUG MARKER: oleg158-server.virtnet: executing load_modules_local [ 1140.955932] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1141.237466] LustreError: 46367:0:(ldlm_lib.c:1132:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 1141.255079] LustreError: 46367:0:(ldlm_lib.c:1132:target_handle_connect()) Skipped 29 previous similar messages [ 1144.985199] Lustre: DEBUG MARKER: oleg158-server.virtnet: executing set_default_debug -1 all [ 1153.542450] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 1157.834341] Lustre: DEBUG MARKER: oleg158-server.virtnet: executing set_default_debug -1 all [ 1160.845867] Lustre: 47488:0:(mgs_llog.c:1345:mgs_modify_param()) MGS: modify general/debug_raw_pointers=Y (mode = 0) failed: rc = -17 [ 1167.538820] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 1173.691529] Lustre: DEBUG MARKER: oleg158-server.virtnet: executing set_default_debug -1 all [ 1182.364723] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 1188.816890] Lustre: DEBUG MARKER: oleg158-server.virtnet: executing set_default_debug -1 all [ 1197.165972] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 1206.540469] Lustre: Setting parameter general.llite.*.xattr_cache=0 in log params [ 1220.372387] LustreError: 49564:0:(obd_class.h:479:obd_check_dev()) Device 10 not setup [ 1220.381792] LustreError: 49564:0:(obd_class.h:479:obd_check_dev()) Skipped 67 previous similar messages [ 1220.585616] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 1220.727289] Lustre: server umount lustre-MDT0000 complete [ 1224.917465] LustreError: 46346:0:(ldlm_lockd.c:2607:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1758738211 with bad export cookie 8933148830719303798 [ 1224.922115] LustreError: MGC192.168.201.158@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 1224.930393] LustreError: 46346:0:(ldlm_lockd.c:2607:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 1225.329203] Lustre: server umount lustre-MDT0001 complete [ 1239.659546] Lustre: server umount lustre-OST0000 complete [ 1253.550982] Lustre: server umount lustre-OST0001 complete [ 1263.442477] Lustre: DEBUG MARKER: == conf-sanity test 77: comma-separated MGS NIDs and failover node NIDs ========================================================== 14:24:09 (1758738249) [ 1265.876188] Lustre: DEBUG MARKER: SKIP: conf-sanity test_77 mixed loopback and real device not working [ 1268.046196] Lustre: DEBUG MARKER: == conf-sanity test 78: run resize2fs on MDT and OST filesystems ========================================================== 14:24:13 (1758738253) [ 1278.409185] Lustre: DEBUG MARKER: oleg158-server.virtnet: executing set_hostid [ 1285.169664] Lustre: DEBUG MARKER: oleg158-server.virtnet: executing load_modules_local [ 1293.270316] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 1298.690077] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 1305.538702] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 1305.615755] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1305.796533] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 1305.839704] Lustre: lustre-MDT0000: new disk, initializing [ 1305.947977] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 1309.667165] Lustre: DEBUG MARKER: oleg158-server.virtnet: executing set_default_debug -1 all [ 1316.958815] Lustre: DEBUG MARKER: oleg158-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 1323.510418] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 1323.584658] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 1323.805714] Lustre: lustre-OST0000: new disk, initializing [ 1323.812319] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 1323.815199] Lustre: Skipped 1 previous similar message [ 1325.458890] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:0:ost [ 1325.463204] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:0:ost] [ 1325.485627] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x240000400 [ 1329.748360] Lustre: DEBUG MARKER: oleg158-server.virtnet: executing set_default_debug -1 all [ 1336.238943] Lustre: DEBUG MARKER: oleg158-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 1338.332322] Lustre: DEBUG MARKER: create test files [ 1371.619018] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 1371.620080] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 1371.635160] Lustre: Skipped 11 previous similar messages [ 1371.638940] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 1371.642195] Lustre: Skipped 19 previous similar messages [ 1381.344143] Lustre: lustre-OST0000 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 1381.705696] Lustre: server umount lustre-OST0000 complete [ 1385.556975] Lustre: server umount lustre-MDT0000 complete [ 1394.633865] Lustre: DEBUG MARKER: oleg158-server.virtnet: executing unload_modules_local [ 1397.183934] Key type lgssc unregistered [ 1397.507279] LNet: 54915:0:(lib-ptl.c:966:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 1397.521403] LNetError: 54915:0:(acceptor.c:263:lnet_acceptor_remove_socket()) Interface ens2 not found [ 1398.571917] LNet: Removed LNI 192.168.201.158@tcp [ 1399.349159] Key type .llcrypt unregistered [ 1399.353429] Key type ._llcrypt unregistered [ 1423.262746] Key type ._llcrypt registered [ 1423.264235] Key type .llcrypt registered [ 1423.341442] Lustre: DEBUG MARKER: oleg158-server.virtnet: executing load_modules_local [ 1424.029376] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 1424.093138] alg: No test for adler32 (adler32-zlib) [ 1425.075066] Lustre: Lustre: Build Version: 2.16.58_53_g2e3f3ed [ 1425.299567] LNet: Added LNI 192.168.201.158@tcp [8/256/0/180] [ 1426.937328] Key type lgssc registered [ 1428.210736] Lustre: Echo OBD driver; http://www.lustre.org/ [ 1434.680564] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 1434.702908] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1436.072225] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 1439.656427] Lustre: DEBUG MARKER: oleg158-server.virtnet: executing set_default_debug -1 all [ 1444.804773] Lustre: DEBUG MARKER: oleg158-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 1450.918890] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 1451.334512] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 1456.237979] Lustre: DEBUG MARKER: oleg158-server.virtnet: executing set_default_debug -1 all [ 1461.439306] Lustre: DEBUG MARKER: oleg158-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 1461.558791] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x240000400:102 to 0x240000400:129) [ 1461.662699] Lustre: 57519:0:(osd_internal.h:1341:osd_trans_exec_op()) lustre-OST0000: opcode 8: before 515 < left 544, rollback = 9 [ 1461.668724] Lustre: 57519:0:(osd_handler.c:2079:osd_trans_dump_creds()) create: 32/128/0, destroy: 0/0/0 [ 1461.672755] Lustre: 57519:0:(osd_handler.c:2086:osd_trans_dump_creds()) attr_set: 0/0/0, xattr_set: 0/0/0 [ 1461.675870] Lustre: 57519:0:(osd_handler.c:2096:osd_trans_dump_creds()) write: 2/13/1, punch: 0/0/0, quota 1/3/0 [ 1461.681510] Lustre: 57519:0:(osd_handler.c:2103:osd_trans_dump_creds()) insert: 32/544/0, delete: 0/0/0 [ 1461.685598] Lustre: 57519:0:(osd_handler.c:2110:osd_trans_dump_creds()) ref_add: 0/0/0, ref_del: 0/0/0 [ 1466.007888] Lustre: DEBUG MARKER: check files after expanding the MDT and OST filesystems [ 1471.273046] Lustre: DEBUG MARKER: create more files after expanding the MDT and OST filesystems [ 1477.094512] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 1477.095618] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 1477.140941] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 1478.914101] LustreError: 58263:0:(obd_class.h:479:obd_check_dev()) Device 13 not setup [ 1479.039897] Lustre: server umount lustre-OST0000 complete [ 1482.375897] LustreError: 58464:0:(obd_class.h:479:obd_check_dev()) Device 8 not setup [ 1482.379468] LustreError: 58464:0:(obd_class.h:479:obd_check_dev()) Skipped 1 previous similar message [ 1482.603660] Lustre: server umount lustre-MDT0000 complete [ 1490.696914] Lustre: DEBUG MARKER: oleg158-server.virtnet: executing unload_modules_local [ 1493.237929] Key type lgssc unregistered [ 1493.541612] LNet: 58996:0:(lib-ptl.c:966:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 1493.546830] LNetError: 58996:0:(acceptor.c:263:lnet_acceptor_remove_socket()) Interface ens2 not found [ 1493.563563] LNet: Removed LNI 192.168.201.158@tcp [ 1494.301741] Key type .llcrypt unregistered [ 1494.303524] Key type ._llcrypt unregistered [ 1519.532923] Key type ._llcrypt registered [ 1519.537710] Key type .llcrypt registered [ 1519.637785] Lustre: DEBUG MARKER: oleg158-server.virtnet: executing load_modules_local [ 1520.509741] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 1520.656067] alg: No test for adler32 (adler32-zlib) [ 1521.758809] Lustre: Lustre: Build Version: 2.16.58_53_g2e3f3ed [ 1522.067500] LNet: Added LNI 192.168.201.158@tcp [8/256/0/180] [ 1523.792185] Key type lgssc registered [ 1524.772885] Lustre: Echo OBD driver; http://www.lustre.org/ [ 1532.400542] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 1532.441952] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1534.065566] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 1538.719559] Lustre: DEBUG MARKER: oleg158-server.virtnet: executing set_default_debug -1 all [ 1545.200794] Lustre: DEBUG MARKER: oleg158-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 1553.122684] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 1553.514417] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 1559.671762] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x240000400:140 to 0x240000400:161) [ 1559.852052] Lustre: 61678:0:(osd_internal.h:1341:osd_trans_exec_op()) lustre-OST0000: opcode 8: before 515 < left 544, rollback = 9 [ 1559.856304] Lustre: 61678:0:(osd_handler.c:2079:osd_trans_dump_creds()) create: 32/128/0, destroy: 0/0/0 [ 1559.860136] Lustre: 61678:0:(osd_handler.c:2086:osd_trans_dump_creds()) attr_set: 0/0/0, xattr_set: 0/0/0 [ 1559.863240] Lustre: 61678:0:(osd_handler.c:2096:osd_trans_dump_creds()) write: 2/13/1, punch: 0/0/0, quota 1/3/0 [ 1559.866381] Lustre: 61678:0:(osd_handler.c:2103:osd_trans_dump_creds()) insert: 32/544/0, delete: 0/0/0 [ 1559.868977] Lustre: 61678:0:(osd_handler.c:2110:osd_trans_dump_creds()) ref_add: 0/0/0, ref_del: 0/0/0 [ 1559.884846] Lustre: DEBUG MARKER: oleg158-server.virtnet: executing set_default_debug -1 all [ 1566.997419] Lustre: DEBUG MARKER: oleg158-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 1572.999688] Lustre: DEBUG MARKER: check files after shrinking the MDT and OST filesystems [ 1585.636499] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 1585.641807] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 1585.657680] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 1588.186044] LustreError: 62373:0:(obd_class.h:479:obd_check_dev()) Device 13 not setup [ 1588.414774] Lustre: server umount lustre-OST0000 complete [ 1592.731906] LustreError: 62575:0:(obd_class.h:479:obd_check_dev()) Device 8 not setup [ 1592.734793] LustreError: 62575:0:(obd_class.h:479:obd_check_dev()) Skipped 1 previous similar message [ 1593.026381] Lustre: server umount lustre-MDT0000 complete [ 1604.994642] Lustre: DEBUG MARKER: oleg158-server.virtnet: executing unload_modules_local [ 1608.092836] Key type lgssc unregistered [ 1608.525145] LNet: 63108:0:(lib-ptl.c:966:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 1608.534758] LNetError: 63108:0:(acceptor.c:263:lnet_acceptor_remove_socket()) Interface ens2 not found [ 1608.563413] LNet: Removed LNI 192.168.201.158@tcp [ 1609.507077] Key type .llcrypt unregistered [ 1609.508988] Key type ._llcrypt unregistered [ 1628.738708] Key type ._llcrypt registered [ 1628.742315] Key type .llcrypt registered [ 1628.853307] Lustre: DEBUG MARKER: oleg158-server.virtnet: executing set_hostid [ 1646.019431] Lustre: DEBUG MARKER: oleg158-server.virtnet: executing load_modules_local [ 1647.153432] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 1647.179182] alg: No test for adler32 (adler32-zlib) [ 1648.336566] Lustre: Lustre: Build Version: 2.16.58_53_g2e3f3ed [ 1648.601930] LNet: Added LNI 192.168.201.158@tcp [8/256/0/180] [ 1650.248180] Key type lgssc registered [ 1651.464995] Lustre: Echo OBD driver; http://www.lustre.org/ [ 1660.590624] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 1667.178776] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 1673.171481] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 1679.431851] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 1687.331156] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 1687.394541] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 1687.420755] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1688.685121] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 1688.738545] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 1688.826840] Lustre: lustre-MDT0000: new disk, initializing [ 1688.922135] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 1688.950667] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 1692.789789] Lustre: DEBUG MARKER: oleg158-server.virtnet: executing set_default_debug -1 all [ 1704.639644] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 1704.728557] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 1704.814722] Lustre: 66237:0:(mgs_llog.c:1345:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 1704.887264] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [ 1704.890164] Lustre: Skipped 1 previous similar message [ 1704.992872] Lustre: lustre-MDT0001: new disk, initializing [ 1705.149530] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 1705.205507] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [ 1705.220206] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 1709.731559] Lustre: DEBUG MARKER: oleg158-server.virtnet: executing set_default_debug -1 all [ 1717.267502] Lustre: DEBUG MARKER: oleg158-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 1720.777253] Lustre: DEBUG MARKER: oleg158-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 1727.045259] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 1727.120349] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 1727.400157] Lustre: lustre-OST0000: new disk, initializing [ 1727.406510] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 1727.531562] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 1732.057406] Lustre: DEBUG MARKER: oleg158-server.virtnet: executing set_default_debug -1 all [ 1737.841795] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [ 1737.856406] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 1738.006308] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 1739.426175] Lustre: DEBUG MARKER: oleg158-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 1744.247177] Lustre: DEBUG MARKER: oleg158-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 [ 1744.432427] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [ 1748.498923] Lustre: DEBUG MARKER: oleg158-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid 50 [ 1748.681881] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [ 1753.575261] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 1753.577165] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 1753.603691] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 1753.611558] Lustre: Skipped 1 previous similar message [ 1756.486094] LustreError: 69178:0:(obd_class.h:479:obd_check_dev()) Device 19 not setup [ 1756.655978] Lustre: server umount lustre-OST0000 complete [ 1761.769203] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 1761.783709] LustreError: Skipped 1 previous similar message [ 1761.791191] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 1761.803576] Lustre: Skipped 1 previous similar message [ 1761.808301] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 1763.808988] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 1763.813032] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 1766.316459] LustreError: 69379:0:(obd_class.h:479:obd_check_dev()) Device 22 not setup [ 1766.326935] LustreError: 69379:0:(obd_class.h:479:obd_check_dev()) Skipped 3 previous similar messages [ 1766.438603] Lustre: server umount lustre-MDT0000 complete [ 1769.749221] LustreError: 66230:0:(ldlm_lockd.c:2607:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1758738755 with bad export cookie 10122389994157364671 [ 1769.760317] LustreError: MGC192.168.201.158@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 1769.763963] LustreError: 66230:0:(ldlm_lockd.c:2607:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 1769.900501] LustreError: 69581:0:(obd_class.h:479:obd_check_dev()) Device 21 not setup [ 1769.912746] LustreError: 69581:0:(obd_class.h:479:obd_check_dev()) Skipped 5 previous similar messages [ 1770.222622] Lustre: server umount lustre-MDT0001 complete [ 1778.731462] Lustre: DEBUG MARKER: == conf-sanity test 79: format MDT/OST without mgs option (should return errors) ========================================================== 14:32:44 (1758738764) [ 1789.750653] Lustre: DEBUG MARKER: oleg158-server.virtnet: executing load_modules_local [ 1803.632080] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 1828.539663] Lustre: DEBUG MARKER: oleg158-server.virtnet: executing set_hostid [ 1836.986114] Lustre: DEBUG MARKER: oleg158-server.virtnet: executing load_modules_local [ 1846.364861] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 1853.268508] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 1859.515951] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 1865.925383] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 1872.528430] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 1872.575285] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1872.808514] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 1872.847546] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 1872.937691] Lustre: lustre-MDT0000: new disk, initializing [ 1873.047174] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 1873.082239] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 1877.013728] Lustre: DEBUG MARKER: oleg158-server.virtnet: executing set_default_debug -1 all [ 1886.444605] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 1886.529304] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 1886.653271] Lustre: 73821:0:(mgs_llog.c:1345:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 1886.712615] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [ 1886.719391] Lustre: Skipped 1 previous similar message [ 1886.790664] Lustre: lustre-MDT0001: new disk, initializing [ 1886.913996] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 1886.969944] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [ 1886.989261] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 1891.355906] Lustre: DEBUG MARKER: oleg158-server.virtnet: executing set_default_debug -1 all [ 1899.377965] Lustre: DEBUG MARKER: oleg158-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 1903.576451] Lustre: DEBUG MARKER: oleg158-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 1909.592387] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 1909.654047] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 1909.889842] Lustre: lustre-OST0000: new disk, initializing [ 1909.899051] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 1909.975430] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 1914.734672] Lustre: DEBUG MARKER: oleg158-server.virtnet: executing set_default_debug -1 all [ 1919.585710] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [ 1919.600716] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 1919.710605] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 1921.124874] Lustre: DEBUG MARKER: oleg158-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 1924.524815] Lustre: DEBUG MARKER: oleg158-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 [ 1924.688692] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [ 1928.130162] Lustre: DEBUG MARKER: oleg158-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid 50 [ 1928.373788] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [ 1930.209280] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 1930.209453] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 1930.219713] LustreError: Skipped 1 previous similar message [ 1930.234634] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 1930.239042] Lustre: Skipped 1 previous similar message [ 1935.333693] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 1935.336360] Lustre: Skipped 2 previous similar messages [ 1936.182114] LustreError: 76760:0:(obd_class.h:479:obd_check_dev()) Device 19 not setup [ 1936.186789] LustreError: 76760:0:(obd_class.h:479:obd_check_dev()) Skipped 7 previous similar messages [ 1936.288396] Lustre: server umount lustre-OST0000 complete [ 1940.453310] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 1940.468678] Lustre: Skipped 2 previous similar messages [ 1945.569500] LustreError: 76961:0:(obd_class.h:479:obd_check_dev()) Device 22 not setup [ 1945.572189] LustreError: 76961:0:(obd_class.h:479:obd_check_dev()) Skipped 3 previous similar messages [ 1945.575163] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 1945.609676] Lustre: Skipped 3 previous similar messages [ 1945.700113] Lustre: server umount lustre-MDT0000 complete [ 1948.846279] LustreError: 73813:0:(ldlm_lockd.c:2607:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1758738935 with bad export cookie 10122389994157365553 [ 1948.854341] LustreError: MGC192.168.201.158@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 1948.863375] LustreError: 73813:0:(ldlm_lockd.c:2607:ldlm_cancel_handler()) Skipped 1 previous similar message [ 1949.258676] Lustre: server umount lustre-MDT0001 complete [ 1956.267559] Lustre: DEBUG MARKER: == conf-sanity test 80: mgc import reconnect race ======== 14:35:42 (1758738942) [ 1961.790845] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1962.282064] LustreError: 77836:0:(ldlm_lib.c:1132:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 1962.290477] LustreError: 77836:0:(ldlm_lib.c:1132:target_handle_connect()) Skipped 3 previous similar messages [ 1962.354128] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 1966.046954] Lustre: DEBUG MARKER: oleg158-server.virtnet: executing set_default_debug -1 all [ 1967.586416] LustreError: 77836:0:(ldlm_lib.c:1132:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 1972.718263] LustreError: 77836:0:(ldlm_lib.c:1132:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 1973.329759] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 1976.593461] Lustre: DEBUG MARKER: oleg158-server.virtnet: executing set_default_debug -1 all [ 1981.279350] Lustre: DEBUG MARKER: oleg158-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 1984.569225] Lustre: DEBUG MARKER: oleg158-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 1991.128653] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 1996.941213] Lustre: DEBUG MARKER: oleg158-server.virtnet: executing set_default_debug -1 all [ 2002.048851] Lustre: DEBUG MARKER: oleg158-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 2005.304647] Lustre: 80176:0:(genops.c:1791:obd_export_evict_by_uuid()) MGS: evicting 084bd27e-217f-46a6-bb7e-1a2ff61ce8a5 at adminstrative request [ 2007.780205] LustreError: MGC192.168.201.158@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 2007.797286] Lustre: *** cfs_fail_loc=906, val=2147483648*** [ 2018.282864] Lustre: MGS: Client 084bd27e-217f-46a6-bb7e-1a2ff61ce8a5 (at 0@lo) reconnecting [ 2018.288944] Lustre: Evicted from MGS (at 0@lo) after server handle changed from 0x8c79f411a7b7dca3 to 0x8c79f411a7b7df43 [ 2018.307219] Lustre: MGC192.168.201.158@tcp: Connection restored to 0@lo (at 0@lo) [ 2042.573514] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 2042.645291] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 2042.811366] Lustre: lustre-OST0001: new disk, initializing [ 2042.815108] Lustre: srv-lustre-OST0001: No data found on store. Initialize space. [ 2042.939480] Lustre: lustre-OST0001: Imperative Recovery not enabled, recovery window 60-180 [ 2042.948145] Lustre: Skipped 2 previous similar messages [ 2048.400828] Lustre: DEBUG MARKER: oleg158-server.virtnet: executing set_default_debug -1 all [ 2049.616068] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x00000002c0000400-0x0000000300000400]:1:ost [ 2049.627588] Lustre: cli-lustre-OST0001-super: Allocated super-sequence [0x00000002c0000400-0x0000000300000400]:1:ost] [ 2049.756557] Lustre: lustre-OST0001-osc-MDT0000: update sequence from 0x100010000 to 0x2c0000401 [ 2055.484398] Lustre: DEBUG MARKER: oleg158-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0001-osc-[-0-9a-f]*.ost_server_uuid [ 2060.257099] LustreError: lustre-OST0001-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 2060.260852] Lustre: lustre-OST0001-osc-MDT0001: Connection to lustre-OST0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2060.268167] LustreError: Skipped 1 previous similar message [ 2060.288757] Lustre: lustre-OST0001: Not available for connect from 0@lo (stopping) [ 2060.295368] Lustre: Skipped 1 previous similar message [ 2063.731095] LustreError: 81313:0:(obd_class.h:479:obd_check_dev()) Device 25 not setup [ 2063.739772] LustreError: 81313:0:(obd_class.h:479:obd_check_dev()) Skipped 13 previous similar messages [ 2063.779841] Lustre: server umount lustre-OST0001 complete [ 2065.384495] LustreError: 79435:0:(ldlm_lib.c:1132:target_handle_connect()) lustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 2065.396203] LustreError: 79435:0:(ldlm_lib.c:1132:target_handle_connect()) Skipped 1 previous similar message [ 2069.473652] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 2070.507514] LustreError: 79435:0:(ldlm_lib.c:1132:target_handle_connect()) lustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 2070.515301] LustreError: 79435:0:(ldlm_lib.c:1132:target_handle_connect()) Skipped 1 previous similar message [ 2072.754563] Lustre: server umount lustre-OST0000 complete [ 2080.741375] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2080.751128] Lustre: Skipped 4 previous similar messages [ 2081.883106] LustreError: 81716:0:(obd_class.h:479:obd_check_dev()) Device 28 not setup [ 2081.886363] LustreError: 81716:0:(obd_class.h:479:obd_check_dev()) Skipped 7 previous similar messages [ 2081.946284] Lustre: server umount lustre-MDT0000 complete [ 2085.047761] LustreError: 80571:0:(ldlm_lockd.c:2607:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1758739071 with bad export cookie 10122389994157367107 [ 2085.048540] LustreError: MGC192.168.201.158@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 2085.054870] LustreError: 80571:0:(ldlm_lockd.c:2607:ldlm_cancel_handler()) Skipped 2 previous similar messages [ 2085.362574] Lustre: server umount lustre-MDT0001 complete [ 2092.118936] Lustre: DEBUG MARKER: == conf-sanity test 81: sparse OST indexing ============== 14:37:58 (1758739078) [ 2093.359322] Lustre: DEBUG MARKER: SKIP: conf-sanity test_81 needs >= 3 OSTs [ 2094.862327] Lustre: DEBUG MARKER: == conf-sanity test 82a: specify OSTs for file (succeed) or directory (succeed) ========================================================== 14:38:00 (1758739080) [ 2096.060713] Lustre: DEBUG MARKER: SKIP: conf-sanity test_82a needs >= 3 OSTs [ 2097.525131] Lustre: DEBUG MARKER: == conf-sanity test 82b: specify OSTs for file with --pool and --ost-list options ========================================================== 14:38:03 (1758739083) [ 2098.688842] Lustre: DEBUG MARKER: SKIP: conf-sanity test_82b needs >= 4 OSTs [ 2100.094993] Lustre: DEBUG MARKER: == conf-sanity test 83: ENOSPACE on OST doesn't cause message VFS: Busy inodes after unmount ... ========================================================== 14:38:06 (1758739086) [ 2101.201260] Lustre: DEBUG MARKER: mount the OST /dev/mapper/ost1_flakey as a ldiskfs filesystem [ 2105.202240] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 2105.970760] loop: module loaded [ 2105.985263] loop0: detected capacity change from 0 to 4294967296 [ 2106.033960] LDISKFS-fs (loop0): mounted filesystem with ordered data mode. Opts: (null) [ 2107.207753] Lustre: DEBUG MARKER: run llverfs in partial mode on the OST ldiskfs /mnt/lustre-ost1 [ 2110.153674] Lustre: DEBUG MARKER: oleg158-server.virtnet: executing run_llverfs /mnt/lustre-ost1 -vpl no [ 2113.793245] Lustre: DEBUG MARKER: unmount the OST /dev/mapper/ost1_flakey [ 2121.922551] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 2125.517885] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 2128.845392] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 2132.529829] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 2139.105329] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 2139.188141] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 2139.238958] LustreError: 84491:0:(obd_config.c:783:class_setup()) setup lustre-OST0000-osd failed (-28) [ 2139.245825] LustreError: 84491:0:(obd_mount.c:193:lustre_start_simple()) lustre-OST0000-osd setup error -28 [ 2139.249334] LustreError: 84491:0:(tgt_mount.c:2447:server_fill_super()) Unable to start osd on /dev/mapper/ost1_flakey: -28 [ 2139.269267] LustreError: 84491:0:(super25.c:199:lustre_fill_super()) llite: Unable to mount : rc = -28 [ 2156.143696] Lustre: DEBUG MARKER: oleg158-server.virtnet: executing set_hostid [ 2164.282950] Lustre: DEBUG MARKER: oleg158-server.virtnet: executing load_modules_local [ 2172.885606] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 2179.041561] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 2184.518310] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 2189.870443] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 2195.444217] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 2195.512876] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 2195.731333] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 2195.771938] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 2195.834896] Lustre: lustre-MDT0000: new disk, initializing [ 2195.911757] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 2195.943104] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 2199.284319] Lustre: DEBUG MARKER: oleg158-server.virtnet: executing set_default_debug -1 all [ 2208.329500] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 2208.384936] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 2208.442781] Lustre: 87333:0:(mgs_llog.c:1345:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 2208.670598] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 2211.681560] Lustre: DEBUG MARKER: oleg158-server.virtnet: executing set_default_debug -1 all [ 2218.160313] Lustre: DEBUG MARKER: oleg158-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 2221.483600] Lustre: DEBUG MARKER: oleg158-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 2226.871836] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 2226.945434] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 2231.976556] Lustre: DEBUG MARKER: oleg158-server.virtnet: executing set_default_debug -1 all [ 2236.449659] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 2236.535310] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 2237.511672] Lustre: DEBUG MARKER: oleg158-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 2240.562818] Lustre: DEBUG MARKER: oleg158-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 [ 2240.750369] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [ 2243.858404] Lustre: DEBUG MARKER: oleg158-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid 50 [ 2244.054457] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [ 2246.625338] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 2246.631604] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2246.637022] LustreError: Skipped 2 previous similar messages [ 2246.640574] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 2246.648037] Lustre: Skipped 7 previous similar messages [ 2251.837097] LustreError: 90269:0:(obd_class.h:479:obd_check_dev()) Device 19 not setup [ 2251.840847] LustreError: 90269:0:(obd_class.h:479:obd_check_dev()) Skipped 17 previous similar messages [ 2251.999414] Lustre: server umount lustre-OST0000 complete [ 2261.646727] Lustre: server umount lustre-MDT0000 complete [ 2261.985847] LustreError: 90271:0:(ldlm_lib.c:1132:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 2262.007269] LustreError: 90271:0:(ldlm_lib.c:1132:target_handle_connect()) Skipped 1 previous similar message [ 2264.486073] LustreError: 89088:0:(ldlm_lockd.c:2607:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1758739250 with bad export cookie 10122389994157367702 [ 2264.496202] LustreError: MGC192.168.201.158@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 2264.721308] Lustre: server umount lustre-MDT0001 complete [ 2273.005737] Lustre: DEBUG MARKER: == conf-sanity test 84: check recovery_hard_time ========= 14:40:58 (1758739258) [ 2278.445714] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 2278.724794] LustreError: 91344:0:(ldlm_lib.c:1132:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 2282.061787] Lustre: DEBUG MARKER: oleg158-server.virtnet: executing set_default_debug -1 all [ 2289.646590] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 2293.519808] Lustre: DEBUG MARKER: oleg158-server.virtnet: executing set_default_debug -1 all [ 2298.240352] Lustre: DEBUG MARKER: oleg158-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 2301.700422] Lustre: DEBUG MARKER: oleg158-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 2307.651725] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 2312.695548] Lustre: DEBUG MARKER: oleg158-server.virtnet: executing set_default_debug -1 all [ 2317.969971] Lustre: DEBUG MARKER: oleg158-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 2322.152821] Lustre: DEBUG MARKER: oleg158-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 [ 2324.534652] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 2 sec [ 2327.999637] Lustre: DEBUG MARKER: oleg158-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid 50 [ 2328.208293] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [ 2335.032257] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 2335.094391] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 2335.267467] Lustre: lustre-OST0001: new disk, initializing [ 2335.269701] Lustre: Skipped 2 previous similar messages [ 2335.278583] Lustre: srv-lustre-OST0001: No data found on store. Initialize space. [ 2335.282412] Lustre: Skipped 3 previous similar messages [ 2335.390633] Lustre: lustre-OST0001: Imperative Recovery not enabled, recovery window 60-180 [ 2335.398634] Lustre: Skipped 5 previous similar messages [ 2340.660884] Lustre: DEBUG MARKER: oleg158-server.virtnet: executing set_default_debug -1 all [ 2344.506844] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x00000002c0000400-0x0000000300000400]:1:ost [ 2344.511343] Lustre: Skipped 2 previous similar messages [ 2344.515645] Lustre: cli-lustre-OST0001-super: Allocated super-sequence [0x00000002c0000400-0x0000000300000400]:1:ost] [ 2344.596443] Lustre: lustre-OST0001-osc-MDT0000: update sequence from 0x100010000 to 0x2c0000401 [ 2346.971393] Lustre: DEBUG MARKER: oleg158-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0001-osc-[-0-9a-f]*.ost_server_uuid [ 2350.360271] Lustre: DEBUG MARKER: oleg158-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0001-osc-MDT0000.ost_server_uuid 50 [ 2350.533637] Lustre: DEBUG MARKER: os[cp].lustre-OST0001-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [ 2353.787729] Lustre: DEBUG MARKER: oleg158-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0001-osc-MDT0001.ost_server_uuid 50 [ 2354.092602] Lustre: DEBUG MARKER: os[cp].lustre-OST0001-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [ 2371.290943] Lustre: DEBUG MARKER: mds1 REPLAY BARRIER on lustre-MDT0000 [ 2398.231487] Lustre: Failing over lustre-MDT0000 [ 2398.614665] LustreError: 96272:0:(obd_class.h:479:obd_check_dev()) Device 28 not setup [ 2398.618797] LustreError: 96272:0:(obd_class.h:479:obd_check_dev()) Skipped 17 previous similar messages [ 2398.672438] Lustre: server umount lustre-MDT0000 complete [ 2401.253588] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2401.257730] LustreError: 92663:0:(ldlm_lib.c:1132:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 2401.261569] Lustre: Skipped 6 previous similar messages [ 2401.271799] LustreError: 92663:0:(ldlm_lib.c:1132:target_handle_connect()) Skipped 6 previous similar messages [ 2417.638053] Lustre: 64449:0:(client.c:2464:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1758739387/real 1758739387] req@0000000003f8eac2 x1844171122132480/t0(0) o400->MGC192.168.201.158@tcp@0@lo:26/25 lens 224/224 e 0 to 1 dl 1758739403 ref 1 fl Rpc:XNQr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 [ 2417.673275] LustreError: MGC192.168.201.158@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 2421.850179] LDISKFS-fs (dm-0): recovery complete [ 2421.853087] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 2427.199236] Lustre: lustre-MDT0000: in recovery but waiting for the first client to connect [ 2427.985099] Lustre: lustre-MDT0000: Will be in recovery for at least 1:00, or until 3 clients reconnect [ 2427.986699] Lustre: 91339:0:(ldlm_lib.c:2020:extend_recovery_timer()) lustre-MDT0000: extended recovery timer reached hard limit: 60, extend: 0 [ 2431.214299] Lustre: DEBUG MARKER: oleg158-server.virtnet: executing set_default_debug -1 all [ 2432.486899] Lustre: 91343:0:(ldlm_lib.c:2020:extend_recovery_timer()) lustre-MDT0000: extended recovery timer reached hard limit: 60, extend: 0 [ 2432.492145] Lustre: 91343:0:(ldlm_lib.c:2020:extend_recovery_timer()) Skipped 1 previous similar message [ 2432.500334] Lustre: lustre-MDT0000-lwp-MDT0001: Connection restored to 0@lo (at 0@lo) [ 2432.567339] LustreError: 97022:0:(ldlm_lib.c:2632:replay_request_or_update()) cfs_fail_timeout id 709 sleeping for 300ms [ 2432.898947] LustreError: 97022:0:(ldlm_lib.c:2632:replay_request_or_update()) cfs_fail_timeout id 709 awake [ 2433.235353] LustreError: 97022:0:(ldlm_lib.c:2632:replay_request_or_update()) cfs_fail_timeout id 709 sleeping for 300ms [ 2433.243095] LustreError: 97022:0:(ldlm_lib.c:2632:replay_request_or_update()) Skipped 1 previous similar message [ 2433.560382] LustreError: 97022:0:(ldlm_lib.c:2632:replay_request_or_update()) cfs_fail_timeout id 709 awake [ 2433.574826] LustreError: 97022:0:(ldlm_lib.c:2632:replay_request_or_update()) Skipped 1 previous similar message [ 2433.583367] Lustre: 97022:0:(ldlm_lib.c:2020:extend_recovery_timer()) lustre-MDT0000: extended recovery timer reached hard limit: 60, extend: 1 [ 2433.596265] Lustre: 97022:0:(ldlm_lib.c:2020:extend_recovery_timer()) Skipped 7 previous similar messages [ 2434.547697] LustreError: 97022:0:(ldlm_lib.c:2632:replay_request_or_update()) cfs_fail_timeout id 709 sleeping for 300ms [ 2434.576102] LustreError: 97022:0:(ldlm_lib.c:2632:replay_request_or_update()) Skipped 3 previous similar messages [ 2434.896470] LustreError: 97022:0:(ldlm_lib.c:2632:replay_request_or_update()) cfs_fail_timeout id 709 awake [ 2434.906813] LustreError: 97022:0:(ldlm_lib.c:2632:replay_request_or_update()) Skipped 3 previous similar messages [ 2435.902686] Lustre: 97022:0:(ldlm_lib.c:2020:extend_recovery_timer()) lustre-MDT0000: extended recovery timer reached hard limit: 60, extend: 1 [ 2435.917155] Lustre: 97022:0:(ldlm_lib.c:2020:extend_recovery_timer()) Skipped 6 previous similar messages [ 2436.567815] LustreError: 97022:0:(ldlm_lib.c:2632:replay_request_or_update()) cfs_fail_timeout id 709 sleeping for 300ms [ 2436.575105] LustreError: 97022:0:(ldlm_lib.c:2632:replay_request_or_update()) Skipped 5 previous similar messages [ 2436.904165] LustreError: 97022:0:(ldlm_lib.c:2632:replay_request_or_update()) cfs_fail_timeout id 709 awake [ 2436.907194] LustreError: 97022:0:(ldlm_lib.c:2632:replay_request_or_update()) Skipped 5 previous similar messages [ 2440.115454] Lustre: 97022:0:(ldlm_lib.c:2020:extend_recovery_timer()) lustre-MDT0000: extended recovery timer reached hard limit: 60, extend: 1 [ 2440.160410] Lustre: 97022:0:(ldlm_lib.c:2020:extend_recovery_timer()) Skipped 12 previous similar messages [ 2440.818500] LustreError: 97022:0:(ldlm_lib.c:2632:replay_request_or_update()) cfs_fail_timeout id 709 sleeping for 300ms [ 2440.826516] LustreError: 97022:0:(ldlm_lib.c:2632:replay_request_or_update()) Skipped 12 previous similar messages [ 2441.145640] LustreError: 97022:0:(ldlm_lib.c:2632:replay_request_or_update()) cfs_fail_timeout id 709 awake [ 2441.155026] LustreError: 97022:0:(ldlm_lib.c:2632:replay_request_or_update()) Skipped 12 previous similar messages [ 2448.180712] Lustre: 97022:0:(ldlm_lib.c:2020:extend_recovery_timer()) lustre-MDT0000: extended recovery timer reached hard limit: 60, extend: 1 [ 2448.189476] Lustre: 97022:0:(ldlm_lib.c:2020:extend_recovery_timer()) Skipped 24 previous similar messages [ 2448.827407] LustreError: 97022:0:(ldlm_lib.c:2632:replay_request_or_update()) cfs_fail_timeout id 709 sleeping for 300ms [ 2448.842101] LustreError: 97022:0:(ldlm_lib.c:2632:replay_request_or_update()) Skipped 24 previous similar messages [ 2449.160150] LustreError: 97022:0:(ldlm_lib.c:2632:replay_request_or_update()) cfs_fail_timeout id 709 awake [ 2449.180956] LustreError: 97022:0:(ldlm_lib.c:2632:replay_request_or_update()) Skipped 24 previous similar messages [ 2464.394396] Lustre: 97022:0:(ldlm_lib.c:2020:extend_recovery_timer()) lustre-MDT0000: extended recovery timer reached hard limit: 60, extend: 1 [ 2464.425772] Lustre: 97022:0:(ldlm_lib.c:2020:extend_recovery_timer()) Skipped 50 previous similar messages [ 2465.065395] LustreError: 97022:0:(ldlm_lib.c:2632:replay_request_or_update()) cfs_fail_timeout id 709 sleeping for 300ms [ 2465.072182] LustreError: 97022:0:(ldlm_lib.c:2632:replay_request_or_update()) Skipped 50 previous similar messages [ 2465.392172] LustreError: 97022:0:(ldlm_lib.c:2632:replay_request_or_update()) cfs_fail_timeout id 709 awake [ 2465.396643] LustreError: 97022:0:(ldlm_lib.c:2632:replay_request_or_update()) Skipped 50 previous similar messages [ 2487.016303] Lustre: 97022:0:(ldlm_lib.c:2328:target_recovery_overseer()) lustre-MDT0000 recovery is aborted by hard timeout [ 2487.030492] Lustre: 97022:0:(ldlm_lib.c:2338:target_recovery_overseer()) recovery is aborted, evict exports in recovery [ 2487.040590] Lustre: 97022:0:(genops.c:1618:class_disconnect_stale_exports()) lustre-MDT0000: disconnect stale client 374c01c2-cfe5-46d3-8417-abbfea78588f@192.168.201.58@tcp [ 2487.050113] Lustre: lustre-MDT0000: disconnecting 1 stale clients [ 2487.053555] Lustre: 97022:0:(ldlm_lib.c:1850:abort_req_replay_queue()) @@@ aborted: req@000000007789642a x1844171106223360/t0(8589934767) o101->374c01c2-cfe5-46d3-8417-abbfea78588f@192.168.201.58@tcp:203/0 lens 592/0 e 0 to 0 dl 1758739483 ref 1 fl Complete:/604/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:0 [ 2487.085045] Lustre: lustre-MDT0000-osd: cancel update llog [0x200000400:0x1:0x0] [ 2487.089806] Lustre: lustre-MDT0000: Denying connection for new client 374c01c2-cfe5-46d3-8417-abbfea78588f (at 192.168.201.58@tcp), waiting for 3 known clients (2 recovered, 0 in progress, and 1 evicted) already passed deadline 0:00 [ 2487.115372] Lustre: lustre-MDT0001-osp-MDT0000: cancel update llog [0x240000401:0x1:0x0] [ 2487.144264] Lustre: lustre-MDT0000-osp-MDT0001: Connection restored to 0@lo (at 0@lo) [ 2487.148487] Lustre: lustre-MDT0000: Recovery over after 1:00, of 3 clients 2 recovered and 1 was evicted. [ 2487.151805] Lustre: Skipped 2 previous similar messages [ 2487.180851] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:76 to 0x280000401:577) [ 2487.208482] Lustre: lustre-OST0001: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x2c0000401:45 to 0x2c0000401:545) [ 2499.553192] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 2499.570395] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 2499.577618] Lustre: Skipped 4 previous similar messages [ 2505.308820] Lustre: server umount lustre-OST0000 complete [ 2509.287463] LustreError: 97383:0:(ldlm_lib.c:1132:target_handle_connect()) lustre-OST0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 2509.305352] LustreError: 97383:0:(ldlm_lib.c:1132:target_handle_connect()) Skipped 37 previous similar messages [ 2514.621261] Lustre: server umount lustre-OST0001 complete [ 2532.320160] Lustre: lustre-MDT0000 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 2532.448827] LustreError: 98085:0:(obd_class.h:479:obd_check_dev()) Device 10 not setup [ 2532.454327] LustreError: 98085:0:(obd_class.h:479:obd_check_dev()) Skipped 15 previous similar messages [ 2532.534924] Lustre: server umount lustre-MDT0000 complete [ 2536.000202] LustreError: 91326:0:(ldlm_lockd.c:2607:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1758739522 with bad export cookie 10122389994157418914 [ 2536.011447] LustreError: MGC192.168.201.158@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 2536.023572] LustreError: 91326:0:(ldlm_lockd.c:2607:ldlm_cancel_handler()) Skipped 5 previous similar messages [ 2536.360030] Lustre: server umount lustre-MDT0001 complete [ 2544.337733] Lustre: DEBUG MARKER: == conf-sanity test 85: osd_ost init: fail ea_fid_set ==== 14:45:30 (1758739530) [ 2550.857510] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 2550.863501] Lustre: *** cfs_fail_loc=197, val=0*** [ 2568.481733] LustreError: Failed to get MGS log lustre-sptlrpc and no local copy. [ 2574.624501] LustreError: 98979:0:(mgc_request.c:233:do_config_log_add()) MGC192.168.201.158@tcp: failed processing log, type 4: rc = -110 [ 2580.803568] Lustre: *** cfs_fail_loc=197, val=0*** [ 2580.815317] Lustre: Skipped 65 previous similar messages [ 2603.424471] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 2603.434405] Lustre: Skipped 1 previous similar message [ 2609.945412] Lustre: DEBUG MARKER: oleg158-server.virtnet: executing set_default_debug -1 all [ 2614.986461] Lustre: DEBUG MARKER: oleg158-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 2617.007781] Lustre: server umount lustre-OST0000 complete [ 2624.871384] Lustre: DEBUG MARKER: == conf-sanity test 86: Replacing mkfs.lustre -G option == 14:46:50 (1758739610) [ 2644.354674] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 2644.882161] LustreError: 100491:0:(ldlm_lib.c:1132:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 2644.903393] LustreError: 100491:0:(ldlm_lib.c:1132:target_handle_connect()) Skipped 4 previous similar messages [ 2648.493957] Lustre: DEBUG MARKER: oleg158-server.virtnet: executing set_default_debug -1 all [ 2655.617857] Lustre: DEBUG MARKER: oleg158-client.virtnet: executing wait_import_state_mount (FULL|IDLE) mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 2658.660529] LustreError: 100521:0:(osp_object.c:617:osp_attr_get()) lustre-MDT0001-osp-MDT0000: osp_attr_get update error [0x200000009:0x1:0x0]: rc = -5 [ 2658.677498] LustreError: 100521:0:(lod_sub_object.c:917:lod_sub_prep_llog()) lustre-MDT0000-mdtlov: can't get id from catalogs: rc = -5 [ 2658.684611] LustreError: 100521:0:(lod_dev.c:510:lod_sub_recovery_thread()) lustre-MDT0001-osp-MDT0000: get update log duration 13, retries 0, failed: rc = -5 [ 2658.946593] Lustre: server umount lustre-MDT0000 complete [ 2669.338664] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 2681.061631] Lustre: DEBUG MARKER: oleg158-server.virtnet: executing set_hostid [ 2688.216558] Lustre: DEBUG MARKER: oleg158-server.virtnet: executing load_modules_local [ 2696.143750] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 2701.778950] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 2707.348352] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 2712.970707] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 2718.644761] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 2718.704580] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 2718.995231] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 2719.017824] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 2719.079441] Lustre: lustre-MDT0000: new disk, initializing [ 2719.193923] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 2722.723195] Lustre: DEBUG MARKER: oleg158-server.virtnet: executing set_default_debug -1 all [ 2732.758447] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 2732.846970] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 2733.046823] Lustre: 104562:0:(mgs_llog.c:1345:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 2733.351929] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 2737.003395] Lustre: DEBUG MARKER: oleg158-server.virtnet: executing set_default_debug -1 all [ 2743.559433] Lustre: DEBUG MARKER: oleg158-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 2746.644701] Lustre: DEBUG MARKER: oleg158-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 2752.589341] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 2752.648756] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 2757.317190] Lustre: DEBUG MARKER: oleg158-server.virtnet: executing set_default_debug -1 all [ 2759.291256] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 2763.534298] Lustre: DEBUG MARKER: oleg158-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 2766.953815] Lustre: DEBUG MARKER: oleg158-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 [ 2767.159038] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [ 2770.509883] Lustre: DEBUG MARKER: oleg158-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid 50 [ 2770.715361] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [ 2774.499588] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2774.499907] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 2774.506030] Lustre: Skipped 6 previous similar messages [ 2774.506763] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 2774.512378] LustreError: Skipped 2 previous similar messages [ 2774.514292] Lustre: Skipped 14 previous similar messages [ 2778.552554] Lustre: server umount lustre-OST0000 complete [ 2788.047791] Lustre: server umount lustre-MDT0000 complete [ 2790.921921] LustreError: 104552:0:(ldlm_lockd.c:2607:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1758739777 with bad export cookie 10122389994157429456 [ 2790.928937] LustreError: MGC192.168.201.158@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 2790.935747] LustreError: 104552:0:(ldlm_lockd.c:2607:ldlm_cancel_handler()) Skipped 1 previous similar message [ 2791.090844] LustreError: 107901:0:(obd_class.h:479:obd_check_dev()) Device 21 not setup [ 2791.100563] LustreError: 107901:0:(obd_class.h:479:obd_check_dev()) Skipped 43 previous similar messages [ 2791.318846] Lustre: server umount lustre-MDT0001 complete [ 2798.791968] Lustre: DEBUG MARKER: == conf-sanity test 87: check if MDT inode can hold EAs with N stripes properly ========================================================== 14:49:44 (1758739784) [ 2805.856184] Lustre: DEBUG MARKER: oleg158-server.virtnet: executing unload_modules_local [ 2808.203657] Key type lgssc unregistered [ 2808.451691] LNet: 108628:0:(lib-ptl.c:966:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 2808.458745] LNetError: 108628:0:(acceptor.c:263:lnet_acceptor_remove_socket()) Interface ens2 not found [ 2808.471790] LNet: Removed LNI 192.168.201.158@tcp [ 2809.089888] Key type .llcrypt unregistered [ 2809.092313] Key type ._llcrypt unregistered [ 2823.046533] Key type ._llcrypt registered [ 2823.049277] Key type .llcrypt registered [ 2823.153468] Lustre: DEBUG MARKER: oleg158-server.virtnet: executing set_hostid [ 2836.161901] Lustre: DEBUG MARKER: oleg158-server.virtnet: executing load_modules_local [ 2837.177415] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 2837.204483] alg: No test for adler32 (adler32-zlib) [ 2838.296387] Lustre: Lustre: Build Version: 2.16.58_53_g2e3f3ed [ 2838.527774] LNet: Added LNI 192.168.201.158@tcp [8/256/0/180] [ 2840.208189] Key type lgssc registered [ 2841.303509] Lustre: Echo OBD driver; http://www.lustre.org/ [ 2849.300716] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 2854.982679] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 2860.274934] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 2866.481469] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 2872.434594] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 2878.869237] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 2878.939910] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 2878.966662] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 2880.251515] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 2880.292227] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 2880.384786] Lustre: lustre-MDT0000: new disk, initializing [ 2880.499271] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 2880.538361] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 2884.438620] Lustre: DEBUG MARKER: oleg158-server.virtnet: executing set_default_debug -1 all [ 2894.019725] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 2894.076988] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 2894.319791] Lustre: lustre-OST0000: new disk, initializing [ 2894.335545] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 2894.343215] Lustre: Skipped 1 previous similar message [ 2894.437804] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 2896.346537] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:0:ost [ 2896.353150] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:0:ost] [ 2896.390700] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x240000400 [ 2899.240158] Lustre: DEBUG MARKER: oleg158-server.virtnet: executing set_default_debug -1 all [ 2907.792990] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 2907.852876] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 2907.952620] Lustre: lustre-OST0001: new disk, initializing [ 2907.955767] Lustre: srv-lustre-OST0001: No data found on store. Initialize space. [ 2908.027294] Lustre: lustre-OST0001: Imperative Recovery not enabled, recovery window 60-180 [ 2912.289798] Lustre: DEBUG MARKER: oleg158-server.virtnet: executing set_default_debug -1 all [ 2913.899613] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:1:ost [ 2913.910365] Lustre: cli-lustre-OST0001-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:1:ost] [ 2913.940407] Lustre: lustre-OST0001-osc-MDT0000: update sequence from 0x100010000 to 0x280000400 [ 2924.005640] Lustre: lustre-MDT0000-lwp-OST0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2924.021159] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 2924.025369] Lustre: Skipped 1 previous similar message [ 2925.683450] LustreError: 114200:0:(obd_class.h:479:obd_check_dev()) Device 17 not setup [ 2925.876838] Lustre: server umount lustre-MDT0000 complete [ 2931.067116] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: (null) [ 2934.833534] LustreError: 113603:0:(ldlm_lockd.c:2607:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1758739921 with bad export cookie 1210626324046666634 [ 2934.841882] LustreError: MGC192.168.201.158@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 2934.843201] LustreError: 113603:0:(ldlm_lockd.c:2607:ldlm_cancel_handler()) Skipped 1 previous similar message [ 2934.890127] LustreError: 114795:0:(obd_class.h:479:obd_check_dev()) Device 12 not setup [ 2934.895992] LustreError: 114795:0:(obd_class.h:479:obd_check_dev()) Skipped 5 previous similar messages [ 2934.937032] Lustre: server umount lustre-OST0000 complete [ 2937.631909] LustreError: 114997:0:(obd_class.h:479:obd_check_dev()) Device 2 not setup [ 2937.635388] LustreError: 114997:0:(obd_class.h:479:obd_check_dev()) Skipped 1 previous similar message [ 2937.777034] Lustre: server umount lustre-OST0001 complete [ 2944.554057] Lustre: DEBUG MARKER: == conf-sanity test 88: check the default mount options can be overridden ========================================================== 14:52:10 (1758739930) [ 2952.246890] Lustre: DEBUG MARKER: oleg158-server.virtnet: executing load_modules_local [ 2958.730131] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 2959.485808] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 2963.869369] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 2964.704902] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 2969.981551] Lustre: DEBUG MARKER: == conf-sanity test 89: check tunefs --param and --erase-param{s} options ========================================================== 14:52:36 (1758739956) [ 2980.116985] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 2981.617576] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 2983.271618] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 2985.064553] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 2985.944211] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 2987.732827] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 3002.482995] Lustre: DEBUG MARKER: oleg158-server.virtnet: executing set_hostid [ 3010.433640] Lustre: DEBUG MARKER: oleg158-server.virtnet: executing load_modules_local [ 3017.586053] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 3022.369262] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 3027.593225] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 3033.280544] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 3039.210312] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 3039.286897] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 3039.496353] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 3039.526934] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 3039.577863] Lustre: lustre-MDT0000: new disk, initializing [ 3039.647085] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 3039.667066] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 3043.840520] Lustre: DEBUG MARKER: oleg158-server.virtnet: executing set_default_debug -1 all [ 3053.211563] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 3053.267221] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 3053.322382] Lustre: 120323:0:(mgs_llog.c:1345:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 3053.360909] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [ 3053.363498] Lustre: Skipped 1 previous similar message [ 3053.411494] Lustre: lustre-MDT0001: new disk, initializing [ 3053.488636] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 3053.523840] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [ 3053.530903] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 3056.930920] Lustre: DEBUG MARKER: oleg158-server.virtnet: executing set_default_debug -1 all [ 3063.229744] Lustre: DEBUG MARKER: oleg158-client.virtnet: executing wait_import_state_mount FULL failover.node=192.0.2.254@tcp0,mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 3066.031564] Lustre: DEBUG MARKER: oleg158-client.virtnet: executing wait_import_state_mount FULL failover.node=192.0.2.254@tcp0,mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 3071.677643] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 3071.729718] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 3071.996756] Lustre: lustre-OST0000: new disk, initializing [ 3071.999906] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 3072.127340] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 3076.641880] Lustre: DEBUG MARKER: oleg158-server.virtnet: executing set_default_debug -1 all [ 3082.259973] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [ 3082.265768] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 3082.353294] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 3082.535602] Lustre: DEBUG MARKER: oleg158-client.virtnet: executing wait_import_state_mount (FULL|IDLE) failover.node=192.0.2.254@tcp0,osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 3085.502102] Lustre: DEBUG MARKER: oleg158-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 [ 3085.664550] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [ 3088.662724] Lustre: DEBUG MARKER: oleg158-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid 50 [ 3088.864274] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [ 3092.455019] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 3092.455175] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 3092.471784] Lustre: Skipped 1 previous similar message [ 3092.476043] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 3092.478956] Lustre: Skipped 1 previous similar message [ 3096.440825] LustreError: 123260:0:(obd_class.h:479:obd_check_dev()) Device 19 not setup [ 3096.465034] LustreError: 123260:0:(obd_class.h:479:obd_check_dev()) Skipped 3 previous similar messages [ 3096.627542] Lustre: server umount lustre-OST0000 complete [ 3099.618364] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 3099.622068] LustreError: Skipped 1 previous similar message [ 3099.624904] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 3099.643381] Lustre: Skipped 1 previous similar message [ 3099.650249] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 3102.689627] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 3102.693898] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 3105.704393] LustreError: 123461:0:(obd_class.h:479:obd_check_dev()) Device 22 not setup [ 3105.708171] LustreError: 123461:0:(obd_class.h:479:obd_check_dev()) Skipped 3 previous similar messages [ 3105.781454] Lustre: server umount lustre-MDT0000 complete [ 3108.656065] LustreError: 120314:0:(ldlm_lockd.c:2607:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1758740094 with bad export cookie 1210626324046667859 [ 3108.659345] LustreError: MGC192.168.201.158@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 3108.664045] LustreError: 120314:0:(ldlm_lockd.c:2607:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 3108.996480] Lustre: server umount lustre-MDT0001 complete [ 3115.964104] Lustre: DEBUG MARKER: == conf-sanity test 90a: check max_mod_rpcs_in_flight is enforced ========================================================== 14:55:01 (1758740101) [ 3122.954745] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 3123.288187] LustreError: 124334:0:(ldlm_lib.c:1132:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 3123.312379] LustreError: 124334:0:(ldlm_lib.c:1132:target_handle_connect()) Skipped 3 previous similar messages [ 3123.345125] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 3126.595763] Lustre: DEBUG MARKER: oleg158-server.virtnet: executing set_default_debug -1 all [ 3128.810111] LustreError: 124334:0:(ldlm_lib.c:1132:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 3132.902042] LustreError: 124334:0:(ldlm_lib.c:1132:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 3133.191459] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 3136.225280] Lustre: DEBUG MARKER: oleg158-server.virtnet: executing set_default_debug -1 all [ 3140.082356] Lustre: DEBUG MARKER: oleg158-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 3142.502519] Lustre: DEBUG MARKER: oleg158-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 3147.687875] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 3152.562925] Lustre: DEBUG MARKER: oleg158-server.virtnet: executing set_default_debug -1 all [ 3156.501261] Lustre: DEBUG MARKER: oleg158-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 3163.951372] Lustre: *** cfs_fail_loc=159, val=0*** [ 3180.129171] Lustre: lustre-MDT0000: Client 62cfa571-df3c-4147-bbc3-7db8c8513d34 (at 192.168.201.58@tcp) reconnecting [ 3181.393743] Lustre: *** cfs_fail_loc=159, val=0*** [ 3181.395475] Lustre: Skipped 5 previous similar messages [ 3197.030557] Lustre: lustre-MDT0000: Client 62cfa571-df3c-4147-bbc3-7db8c8513d34 (at 192.168.201.58@tcp) reconnecting [ 3200.490515] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 3200.495629] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 3200.505713] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 3200.508533] Lustre: Skipped 1 previous similar message [ 3204.580735] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 3206.475101] LustreError: 126820:0:(obd_class.h:479:obd_check_dev()) Device 21 not setup [ 3206.485540] LustreError: 126820:0:(obd_class.h:479:obd_check_dev()) Skipped 13 previous similar messages [ 3206.598243] Lustre: server umount lustre-OST0000 complete [ 3209.702042] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 3209.710137] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 3209.725868] Lustre: Skipped 2 previous similar messages [ 3209.731254] Lustre: Skipped 2 previous similar messages [ 3215.572950] Lustre: server umount lustre-MDT0000 complete [ 3218.176752] LustreError: 124314:0:(ldlm_lockd.c:2607:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1758740204 with bad export cookie 1210626324046668741 [ 3218.182851] LustreError: MGC192.168.201.158@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 3218.188700] LustreError: 124314:0:(ldlm_lockd.c:2607:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 3218.454419] Lustre: server umount lustre-MDT0001 complete [ 3225.238038] Lustre: DEBUG MARKER: oleg158-server.virtnet: executing unload_modules_local [ 3227.586780] Key type lgssc unregistered [ 3227.844369] LNet: 127756:0:(lib-ptl.c:966:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 3227.850211] LNetError: 127756:0:(acceptor.c:263:lnet_acceptor_remove_socket()) Interface ens2 not found [ 3227.867503] LNet: Removed LNI 192.168.201.158@tcp [ 3228.578207] Key type .llcrypt unregistered [ 3228.580864] Key type ._llcrypt unregistered [ 3247.239710] Key type ._llcrypt registered [ 3247.242112] Key type .llcrypt registered [ 3247.329679] Lustre: DEBUG MARKER: oleg158-server.virtnet: executing load_modules_local [ 3248.073530] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 3248.139652] alg: No test for adler32 (adler32-zlib) [ 3249.130352] Lustre: Lustre: Build Version: 2.16.58_53_g2e3f3ed [ 3249.363309] LNet: Added LNI 192.168.201.158@tcp [8/256/0/180] [ 3251.024211] Key type lgssc registered [ 3251.983701] Lustre: Echo OBD driver; http://www.lustre.org/ [ 3258.319717] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 3258.337577] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 3259.692772] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 3262.353578] Lustre: DEBUG MARKER: oleg158-server.virtnet: executing set_default_debug -1 all [ 3268.835787] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 3269.089945] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 3271.801334] Lustre: DEBUG MARKER: oleg158-server.virtnet: executing set_default_debug -1 all [ 3275.626388] Lustre: DEBUG MARKER: oleg158-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 3278.774853] Lustre: DEBUG MARKER: oleg158-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 3285.014808] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 3285.416520] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 3286.451565] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:42 to 0x280000401:65) [ 3289.728482] Lustre: DEBUG MARKER: oleg158-server.virtnet: executing set_default_debug -1 all [ 3294.338826] Lustre: DEBUG MARKER: oleg158-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 3301.701908] Lustre: *** cfs_fail_loc=159, val=0*** [ 3318.393814] Lustre: lustre-MDT0000: Client ce2d188e-8e68-4cdb-80c4-2cc87db46fc8 (at 192.168.201.58@tcp) reconnecting [ 3320.843916] Lustre: *** cfs_fail_loc=159, val=0*** [ 3320.845925] Lustre: Skipped 2 previous similar messages [ 3337.337118] Lustre: lustre-MDT0001: Client ce2d188e-8e68-4cdb-80c4-2cc87db46fc8 (at 192.168.201.58@tcp) reconnecting [ 3338.853641] Lustre: *** cfs_fail_loc=159, val=0*** [ 3338.855684] Lustre: Skipped 2 previous similar messages [ 3355.246820] Lustre: lustre-MDT0001: Client ce2d188e-8e68-4cdb-80c4-2cc87db46fc8 (at 192.168.201.58@tcp) reconnecting [ 3358.802601] Lustre: 131834:0:(mdt_handler.c:64:mdt_max_mod_rpcs_per_client_set()) max_mod_rpcs_per_client is deprecated, set mdt.*.max_mod_rpcs_in_flight parameter instead [ 3361.316288] Lustre: *** cfs_fail_loc=159, val=0*** [ 3361.318269] Lustre: Skipped 2 previous similar messages [ 3376.743707] Lustre: lustre-MDT0000: Client b33de102-ccc3-4d52-9d77-48c9854f90aa (at 192.168.201.58@tcp) reconnecting [ 3378.271995] Lustre: *** cfs_fail_loc=159, val=0*** [ 3378.279795] Lustre: Skipped 18 previous similar messages [ 3394.656489] Lustre: lustre-MDT0000: Client b33de102-ccc3-4d52-9d77-48c9854f90aa (at 192.168.201.58@tcp) reconnecting [ 3395.937747] Lustre: 132125:0:(mdt_handler.c:64:mdt_max_mod_rpcs_per_client_set()) max_mod_rpcs_per_client is deprecated, set mdt.*.max_mod_rpcs_in_flight parameter instead [ 3401.193401] LustreError: lustre-OST0000-osc-MDT0000: operation ost_destroy to node 0@lo failed: rc = -107 [ 3401.196895] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 3401.197808] LustreError: Skipped 2 previous similar messages [ 3401.207295] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 3401.716133] LustreError: 132222:0:(obd_class.h:479:obd_check_dev()) Device 21 not setup [ 3401.794927] Lustre: server umount lustre-OST0000 complete [ 3407.329734] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 3407.333636] LustreError: Skipped 2 previous similar messages [ 3407.335466] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 3407.341550] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 3407.345940] Lustre: Skipped 1 previous similar message [ 3407.352711] Lustre: Skipped 1 previous similar message [ 3410.326991] LustreError: 132424:0:(obd_class.h:479:obd_check_dev()) Device 9 not setup [ 3410.332289] LustreError: 132424:0:(obd_class.h:479:obd_check_dev()) Skipped 3 previous similar messages [ 3410.370457] Lustre: server umount lustre-MDT0000 complete [ 3412.450666] LustreError: 132006:0:(ldlm_lib.c:1132:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 3412.478390] LustreError: 132006:0:(ldlm_lib.c:1132:target_handle_connect()) Skipped 3 previous similar messages [ 3412.848499] LustreError: 132074:0:(ldlm_lockd.c:2607:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1758740399 with bad export cookie 9635842260565801970 [ 3412.850733] LustreError: MGC192.168.201.158@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 3412.856417] LustreError: 132074:0:(ldlm_lockd.c:2607:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 3423.205233] LustreError: 132625:0:(obd_class.h:479:obd_check_dev()) Device 16 not setup [ 3423.209577] LustreError: 132625:0:(obd_class.h:479:obd_check_dev()) Skipped 5 previous similar messages [ 3423.335086] Lustre: server umount lustre-MDT0001 complete [ 3431.613454] Lustre: DEBUG MARKER: oleg158-server.virtnet: executing unload_modules_local [ 3433.544206] Key type lgssc unregistered [ 3433.767152] LNet: 133158:0:(lib-ptl.c:966:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 3433.774193] LNetError: 133158:0:(acceptor.c:263:lnet_acceptor_remove_socket()) Interface ens2 not found [ 3433.790327] LNet: Removed LNI 192.168.201.158@tcp [ 3434.380530] Key type .llcrypt unregistered [ 3434.382872] Key type ._llcrypt unregistered [ 3451.289805] Key type ._llcrypt registered [ 3451.291701] Key type .llcrypt registered [ 3451.379698] Lustre: DEBUG MARKER: oleg158-server.virtnet: executing load_modules_local [ 3452.335325] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 3452.374427] alg: No test for adler32 (adler32-zlib) [ 3453.370785] Lustre: Lustre: Build Version: 2.16.58_53_g2e3f3ed [ 3453.592382] LNet: Added LNI 192.168.201.158@tcp [8/256/0/180] [ 3455.272697] Key type lgssc registered [ 3456.216702] Lustre: Echo OBD driver; http://www.lustre.org/ [ 3462.296938] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 3462.313169] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 3463.788220] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 3466.881519] Lustre: DEBUG MARKER: oleg158-server.virtnet: executing set_default_debug -1 all [ 3473.378268] LustreError: 134535:0:(ldlm_lib.c:1132:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 3473.396988] LustreError: 134535:0:(ldlm_lib.c:1132:target_handle_connect()) Skipped 2 previous similar messages [ 3473.549718] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 3473.856761] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 3477.050475] Lustre: DEBUG MARKER: oleg158-server.virtnet: executing set_default_debug -1 all [ 3481.722981] Lustre: DEBUG MARKER: oleg158-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 3484.376205] Lustre: DEBUG MARKER: oleg158-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 3489.610469] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 3489.882868] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 3494.155444] Lustre: DEBUG MARKER: oleg158-server.virtnet: executing set_default_debug -1 all [ 3497.850311] Lustre: DEBUG MARKER: oleg158-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 3499.131616] Lustre: lustre-OST0000: new connection from lustre-MDT0001-mdtlov (cleaning up unused objects from 0x280000400:72 to 0x280000400:97) [ 3508.370308] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:85 to 0x280000401:129) [ 3513.178469] Lustre: 136820:0:(mdt_handler.c:64:mdt_max_mod_rpcs_per_client_set()) max_mod_rpcs_per_client is deprecated, set mdt.*.max_mod_rpcs_in_flight parameter instead [ 3514.862155] Lustre: 136869:0:(mdt_handler.c:64:mdt_max_mod_rpcs_per_client_set()) max_mod_rpcs_per_client is deprecated, set mdt.*.max_mod_rpcs_in_flight parameter instead [ 3515.926155] Lustre: Modifying parameter lustre.mdc.lustre-MDT0000-mdc-*.max_rpcs_in_flight=8 in log params [ 3516.545828] Lustre: Setting parameter lustre.mdc.lustre-MDT0000-mdc-*.max_mod_rpcs_in_flight=8 in log params [ 3518.512261] Lustre: 137019:0:(mgs_llog.c:1345:mgs_modify_param()) MGS: modify lustre/mdc.lustre-MDT0000-mdc-*.max_rpcs_in_flight=8 (mode = 0) failed: rc = -17 [ 3519.233266] Lustre: Modifying parameter lustre.mdc.lustre-MDT0000-mdc-*.max_mod_rpcs_in_flight=7 in log params [ 3519.945727] Lustre: 137119:0:(mdt_handler.c:64:mdt_max_mod_rpcs_per_client_set()) max_mod_rpcs_per_client is deprecated, set mdt.*.max_mod_rpcs_in_flight parameter instead [ 3524.069125] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 3524.071118] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 3524.072558] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 3524.075435] Lustre: Skipped 1 previous similar message [ 3528.692089] LustreError: 137216:0:(obd_class.h:479:obd_check_dev()) Device 21 not setup [ 3528.866545] Lustre: server umount lustre-OST0000 complete [ 3534.314490] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 3534.319936] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 3534.325211] Lustre: Skipped 1 previous similar message [ 3534.332129] Lustre: Skipped 2 previous similar messages [ 3537.608230] LustreError: 137424:0:(obd_class.h:479:obd_check_dev()) Device 9 not setup [ 3537.611216] LustreError: 137424:0:(obd_class.h:479:obd_check_dev()) Skipped 3 previous similar messages [ 3537.675499] Lustre: server umount lustre-MDT0000 complete [ 3539.426635] LustreError: 134536:0:(ldlm_lib.c:1132:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 3539.436699] LustreError: 134536:0:(ldlm_lib.c:1132:target_handle_connect()) Skipped 1 previous similar message [ 3540.338267] LustreError: 134515:0:(ldlm_lockd.c:2607:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1758740526 with bad export cookie 15159335991055075948 [ 3540.342158] LustreError: MGC192.168.201.158@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 3540.349021] LustreError: 134515:0:(ldlm_lockd.c:2607:ldlm_cancel_handler()) Skipped 2 previous similar messages [ 3540.443591] LustreError: 137626:0:(obd_class.h:479:obd_check_dev()) Device 16 not setup [ 3540.450359] LustreError: 137626:0:(obd_class.h:479:obd_check_dev()) Skipped 5 previous similar messages [ 3540.602196] Lustre: server umount lustre-MDT0001 complete [ 3548.036272] Lustre: DEBUG MARKER: oleg158-server.virtnet: executing unload_modules_local [ 3550.248948] Key type lgssc unregistered [ 3550.490994] LNet: 138157:0:(lib-ptl.c:966:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 3550.497546] LNetError: 138157:0:(acceptor.c:263:lnet_acceptor_remove_socket()) Interface ens2 not found [ 3550.524942] LNet: Removed LNI 192.168.201.158@tcp [ 3551.126603] Key type .llcrypt unregistered [ 3551.135897] Key type ._llcrypt unregistered [ 3566.710166] Key type ._llcrypt registered [ 3566.711868] Key type .llcrypt registered [ 3566.783718] Lustre: DEBUG MARKER: oleg158-server.virtnet: executing load_modules_local [ 3567.438167] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 3567.450152] alg: No test for adler32 (adler32-zlib) [ 3568.395962] Lustre: Lustre: Build Version: 2.16.58_53_g2e3f3ed [ 3568.559745] LNet: Added LNI 192.168.201.158@tcp [8/256/0/180] [ 3570.200196] Key type lgssc registered [ 3570.975927] Lustre: Echo OBD driver; http://www.lustre.org/ [ 3575.921375] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 3575.939954] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 3577.230545] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 3579.602394] Lustre: DEBUG MARKER: oleg158-server.virtnet: executing set_default_debug -1 all [ 3585.597771] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 3585.852101] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 3588.368606] Lustre: DEBUG MARKER: oleg158-server.virtnet: executing set_default_debug -1 all [ 3591.673308] Lustre: DEBUG MARKER: oleg158-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 3594.029470] Lustre: DEBUG MARKER: oleg158-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 3598.412742] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 3598.746910] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 3601.835351] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:85 to 0x280000401:161) [ 3602.070111] Lustre: DEBUG MARKER: oleg158-server.virtnet: executing set_default_debug -1 all [ 3605.236086] Lustre: DEBUG MARKER: oleg158-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 3611.054832] Lustre: lustre-OST0000: new connection from lustre-MDT0001-mdtlov (cleaning up unused objects from 0x280000400:72 to 0x280000400:129) [ 3614.005416] Lustre: *** cfs_fail_loc=159, val=0*** [ 3630.164532] Lustre: lustre-MDT0000: Client a9e9370a-1b4c-4b00-a257-bdab5c2c94a8 (at 192.168.201.58@tcp) reconnecting [ 3646.950408] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 3646.959578] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 3646.964493] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 3652.074556] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 3652.077409] Lustre: Skipped 2 previous similar messages [ 3652.285188] LustreError: 141913:0:(obd_class.h:479:obd_check_dev()) Device 21 not setup [ 3652.374175] Lustre: server umount lustre-OST0000 complete [ 3657.187476] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 3657.195447] Lustre: Skipped 1 previous similar message [ 3657.199821] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 3657.204191] Lustre: Skipped 1 previous similar message [ 3660.845799] LustreError: 142116:0:(obd_class.h:479:obd_check_dev()) Device 9 not setup [ 3660.849052] LustreError: 142116:0:(obd_class.h:479:obd_check_dev()) Skipped 3 previous similar messages [ 3660.888562] Lustre: server umount lustre-MDT0000 complete [ 3662.305373] LustreError: 141745:0:(ldlm_lib.c:1132:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 3662.318763] LustreError: 141745:0:(ldlm_lib.c:1132:target_handle_connect()) Skipped 3 previous similar messages [ 3663.057826] LustreError: 139513:0:(ldlm_lockd.c:2607:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1758740649 with bad export cookie 10594796464292703489 [ 3663.059285] LustreError: MGC192.168.201.158@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 3663.069278] LustreError: 139513:0:(ldlm_lockd.c:2607:ldlm_cancel_handler()) Skipped 1 previous similar message [ 3663.168042] LustreError: 142318:0:(obd_class.h:479:obd_check_dev()) Device 16 not setup [ 3663.171632] LustreError: 142318:0:(obd_class.h:479:obd_check_dev()) Skipped 5 previous similar messages [ 3663.307709] Lustre: server umount lustre-MDT0001 complete [ 3669.460972] Lustre: DEBUG MARKER: oleg158-server.virtnet: executing unload_modules_local [ 3670.864074] Key type lgssc unregistered [ 3671.032743] LNet: 142849:0:(lib-ptl.c:966:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 3671.037850] LNetError: 142849:0:(acceptor.c:263:lnet_acceptor_remove_socket()) Interface ens2 not found [ 3671.053725] LNet: Removed LNI 192.168.201.158@tcp [ 3671.494849] Key type .llcrypt unregistered [ 3671.496436] Key type ._llcrypt unregistered [ 3683.912496] Key type ._llcrypt registered [ 3683.913944] Key type .llcrypt registered [ 3683.971775] Lustre: DEBUG MARKER: oleg158-server.virtnet: executing load_modules_local [ 3684.385896] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 3684.442448] alg: No test for adler32 (adler32-zlib) [ 3685.345398] Lustre: Lustre: Build Version: 2.16.58_53_g2e3f3ed [ 3685.462949] LNet: Added LNI 192.168.201.158@tcp [8/256/0/180] [ 3687.064192] Key type lgssc registered [ 3687.617200] Lustre: Echo OBD driver; http://www.lustre.org/ [ 3691.596114] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 3691.609785] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 3692.852896] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 3694.922630] Lustre: DEBUG MARKER: oleg158-server.virtnet: executing set_default_debug -1 all [ 3699.179478] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 3699.430333] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 3701.542979] Lustre: DEBUG MARKER: oleg158-server.virtnet: executing set_default_debug -1 all [ 3703.998951] Lustre: DEBUG MARKER: oleg158-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 3705.737541] Lustre: DEBUG MARKER: oleg158-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 3708.820390] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 3709.021409] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 3710.056268] Lustre: lustre-OST0000: new connection from lustre-MDT0001-mdtlov (cleaning up unused objects from 0x280000400:72 to 0x280000400:161) [ 3711.421590] Lustre: DEBUG MARKER: oleg158-server.virtnet: executing set_default_debug -1 all [ 3713.999688] Lustre: DEBUG MARKER: oleg158-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 3718.254992] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:170 to 0x280000401:193) [ 3734.241547] Lustre: lustre-OST0000: evicting 05062009-538c-417a-8c68-179b65f95187 (at 192.168.201.58@tcp) by administrative request [ 3734.246468] Lustre: Skipped 1 previous similar message [ 3761.634679] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 3761.640558] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 3761.655365] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 3764.710019] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 3764.710671] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 3764.711320] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 3764.727772] Lustre: Skipped 1 previous similar message [ 3766.195325] LustreError: 147002:0:(obd_class.h:479:obd_check_dev()) Device 21 not setup [ 3766.329981] Lustre: server umount lustre-OST0000 complete [ 3769.826777] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 3769.834762] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 3769.838544] Lustre: Skipped 1 previous similar message [ 3774.799887] LustreError: 147204:0:(obd_class.h:479:obd_check_dev()) Device 9 not setup [ 3774.802537] LustreError: 147204:0:(obd_class.h:479:obd_check_dev()) Skipped 3 previous similar messages [ 3774.859459] Lustre: server umount lustre-MDT0000 complete [ 3774.947921] LustreError: 144222:0:(ldlm_lib.c:1132:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 3774.955274] LustreError: 144222:0:(ldlm_lib.c:1132:target_handle_connect()) Skipped 3 previous similar messages [ 3777.105300] LustreError: 144203:0:(ldlm_lockd.c:2607:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1758740763 with bad export cookie 5432188623564498302 [ 3777.106858] LustreError: MGC192.168.201.158@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 3777.111634] LustreError: 144203:0:(ldlm_lockd.c:2607:ldlm_cancel_handler()) Skipped 1 previous similar message [ 3777.224115] LustreError: 147405:0:(obd_class.h:479:obd_check_dev()) Device 16 not setup [ 3777.227410] LustreError: 147405:0:(obd_class.h:479:obd_check_dev()) Skipped 5 previous similar messages [ 3777.351406] Lustre: server umount lustre-MDT0001 complete [ 3782.713338] Lustre: DEBUG MARKER: oleg158-server.virtnet: executing unload_modules_local [ 3784.419051] Key type lgssc unregistered [ 3784.617197] LNet: 147935:0:(lib-ptl.c:966:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 3784.623071] LNetError: 147935:0:(acceptor.c:263:lnet_acceptor_remove_socket()) Interface ens2 not found [ 3784.647661] LNet: Removed LNI 192.168.201.158@tcp [ 3785.173660] Key type .llcrypt unregistered [ 3785.175497] Key type ._llcrypt unregistered [ 3826.865470] Key type ._llcrypt registered [ 3826.867204] Key type .llcrypt registered [ 3826.932885] Lustre: DEBUG MARKER: oleg158-server.virtnet: executing load_modules_local [ 3827.529229] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 3827.544748] alg: No test for adler32 (adler32-zlib) [ 3828.487106] Lustre: Lustre: Build Version: 2.16.58_53_g2e3f3ed [ 3828.649535] LNet: Added LNI 192.168.201.158@tcp [8/256/0/180] [ 3830.256305] Key type lgssc registered [ 3830.917758] Lustre: Echo OBD driver; http://www.lustre.org/ [ 3835.434159] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 3835.452178] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 3836.743690] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 3839.088670] Lustre: DEBUG MARKER: oleg158-server.virtnet: executing set_default_debug -1 all [ 3844.061815] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 3844.298289] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 3846.475974] Lustre: DEBUG MARKER: oleg158-server.virtnet: executing set_default_debug -1 all [ 3849.747244] Lustre: DEBUG MARKER: oleg158-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 3851.727208] Lustre: DEBUG MARKER: oleg158-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 3855.349904] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 3855.577692] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 3858.684803] Lustre: DEBUG MARKER: oleg158-server.virtnet: executing set_default_debug -1 all [ 3861.684276] Lustre: DEBUG MARKER: oleg158-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 3861.741901] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:195 to 0x280000401:225) [ 3868.911241] Lustre: lustre-OST0000: new connection from lustre-MDT0001-mdtlov (cleaning up unused objects from 0x280000400:72 to 0x280000400:193) [ 3877.346487] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 3877.349853] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 3877.356718] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 3879.394788] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 3879.397822] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 3879.399053] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 3882.168342] LustreError: 152656:0:(obd_class.h:479:obd_check_dev()) Device 21 not setup [ 3882.316280] Lustre: server umount lustre-OST0000 complete [ 3884.513850] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 3884.518909] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 3884.526979] Lustre: Skipped 1 previous similar message [ 3884.535088] Lustre: Skipped 2 previous similar messages [ 3889.636147] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 3890.571241] LustreError: 152858:0:(obd_class.h:479:obd_check_dev()) Device 9 not setup [ 3890.574237] LustreError: 152858:0:(obd_class.h:479:obd_check_dev()) Skipped 3 previous similar messages [ 3890.619313] Lustre: server umount lustre-MDT0000 complete [ 3892.566268] LustreError: 150359:0:(ldlm_lockd.c:2607:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1758740878 with bad export cookie 17894121323417798884 [ 3892.573129] LustreError: MGC192.168.201.158@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 3892.574856] LustreError: 150359:0:(ldlm_lockd.c:2607:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 3892.659538] LustreError: 153059:0:(obd_class.h:479:obd_check_dev()) Device 16 not setup [ 3892.662576] LustreError: 153059:0:(obd_class.h:479:obd_check_dev()) Skipped 5 previous similar messages [ 3892.795303] Lustre: server umount lustre-MDT0001 complete [ 3899.040450] Lustre: DEBUG MARKER: oleg158-server.virtnet: executing unload_modules_local [ 3900.755786] Key type lgssc unregistered [ 3900.971268] LNet: 153593:0:(lib-ptl.c:966:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 3900.979288] LNetError: 153593:0:(acceptor.c:263:lnet_acceptor_remove_socket()) Interface ens2 not found [ 3900.995957] LNet: Removed LNI 192.168.201.158@tcp [ 3901.484665] Key type .llcrypt unregistered [ 3901.486491] Key type ._llcrypt unregistered [ 3916.381064] Key type ._llcrypt registered [ 3916.382859] Key type .llcrypt registered [ 3916.462392] Lustre: DEBUG MARKER: oleg158-server.virtnet: executing load_modules_local [ 3916.931451] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 3916.991568] alg: No test for adler32 (adler32-zlib) [ 3917.938553] Lustre: Lustre: Build Version: 2.16.58_53_g2e3f3ed [ 3918.076565] LNet: Added LNI 192.168.201.158@tcp [8/256/0/180] [ 3919.688187] Key type lgssc registered [ 3920.444970] Lustre: Echo OBD driver; http://www.lustre.org/ [ 3925.104074] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 3929.688318] Lustre: DEBUG MARKER: == conf-sanity test 100: check lshowmount lists MGS, MDT, OST and 0@lo ========================================================== 15:08:36 (1758740916) [ 3940.011086] Lustre: DEBUG MARKER: oleg158-server.virtnet: executing set_hostid [ 3946.531212] Lustre: DEBUG MARKER: oleg158-server.virtnet: executing load_modules_local [ 3953.231262] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 3957.763796] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 3961.879101] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 3965.774703] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 3969.862646] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 3969.909172] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 3969.925699] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 3971.084886] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 3971.104966] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 3971.148870] Lustre: lustre-MDT0000: new disk, initializing [ 3971.201812] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 3971.214406] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 3973.713851] Lustre: DEBUG MARKER: oleg158-server.virtnet: executing set_default_debug -1 all [ 3980.089626] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 3980.134349] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 3980.186751] Lustre: 157790:0:(mgs_llog.c:1345:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 3980.228577] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [ 3980.232692] Lustre: Skipped 1 previous similar message [ 3980.282025] Lustre: lustre-MDT0001: new disk, initializing [ 3980.348798] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 3980.385410] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [ 3980.402171] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 3982.645895] Lustre: DEBUG MARKER: oleg158-server.virtnet: executing set_default_debug -1 all [ 3987.397727] Lustre: DEBUG MARKER: oleg158-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 3989.310152] Lustre: DEBUG MARKER: oleg158-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 3992.755799] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 3992.791845] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 3992.931441] Lustre: lustre-OST0000: new disk, initializing [ 3992.935791] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 3992.992756] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 3995.753222] Lustre: DEBUG MARKER: oleg158-server.virtnet: executing set_default_debug -1 all [ 3999.891577] Lustre: DEBUG MARKER: oleg158-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 4001.821100] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [ 4001.829186] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 4001.886861] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 4012.001390] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 4012.004169] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4012.005532] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 4012.006515] LustreError: Skipped 1 previous similar message [ 4012.014327] Lustre: Skipped 1 previous similar message [ 4016.751259] LustreError: 160378:0:(obd_class.h:479:obd_check_dev()) Device 19 not setup [ 4016.834586] Lustre: server umount lustre-OST0000 complete [ 4021.217636] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 4021.224387] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4021.232121] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 4021.234618] Lustre: Skipped 1 previous similar message [ 4022.241287] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4022.242623] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 4022.248586] Lustre: Skipped 1 previous similar message [ 4024.716981] LustreError: 160579:0:(obd_class.h:479:obd_check_dev()) Device 22 not setup [ 4024.719603] LustreError: 160579:0:(obd_class.h:479:obd_check_dev()) Skipped 3 previous similar messages [ 4024.763317] Lustre: server umount lustre-MDT0000 complete [ 4026.761668] LustreError: 157782:0:(ldlm_lockd.c:2607:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1758741013 with bad export cookie 11337020762925309232 [ 4026.765595] LustreError: MGC192.168.201.158@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 4026.768225] LustreError: 157782:0:(ldlm_lockd.c:2607:ldlm_cancel_handler()) Skipped 2 previous similar messages [ 4026.854591] LustreError: 160781:0:(obd_class.h:479:obd_check_dev()) Device 21 not setup [ 4026.857567] LustreError: 160781:0:(obd_class.h:479:obd_check_dev()) Skipped 5 previous similar messages [ 4026.983862] Lustre: server umount lustre-MDT0001 complete [ 4032.330694] Lustre: DEBUG MARKER: oleg158-server.virtnet: executing unload_modules_local [ 4033.879329] Key type lgssc unregistered [ 4034.077603] LNet: 161312:0:(lib-ptl.c:966:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 4034.083330] LNetError: 161312:0:(acceptor.c:263:lnet_acceptor_remove_socket()) Interface ens2 not found [ 4034.096057] LNet: Removed LNI 192.168.201.158@tcp [ 4034.669038] Key type .llcrypt unregistered [ 4034.670990] Key type ._llcrypt unregistered [ 4047.884164] Key type ._llcrypt registered [ 4047.885704] Key type .llcrypt registered [ 4047.960079] Lustre: DEBUG MARKER: oleg158-server.virtnet: executing load_modules_local [ 4048.504264] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 4048.520642] alg: No test for adler32 (adler32-zlib) [ 4049.482358] Lustre: Lustre: Build Version: 2.16.58_53_g2e3f3ed [ 4049.606749] LNet: Added LNI 192.168.201.158@tcp [8/256/0/180] [ 4051.264926] Key type lgssc registered [ 4051.916388] Lustre: Echo OBD driver; http://www.lustre.org/ [ 4056.779000] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 4056.800704] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 4058.104644] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 4060.349655] Lustre: DEBUG MARKER: oleg158-server.virtnet: executing set_default_debug -1 all [ 4064.640633] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 4064.850690] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 4067.010970] Lustre: DEBUG MARKER: oleg158-server.virtnet: executing set_default_debug -1 all [ 4070.592916] Lustre: DEBUG MARKER: oleg158-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 4072.890739] Lustre: DEBUG MARKER: oleg158-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 4077.065570] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 4077.316054] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 4080.675740] Lustre: DEBUG MARKER: oleg158-server.virtnet: executing set_default_debug -1 all [ 4083.515164] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:3 to 0x280000401:33) [ 4083.927634] Lustre: DEBUG MARKER: oleg158-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 4092.707333] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 4092.717655] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4092.725242] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 4092.729616] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4092.735953] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 4093.253676] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 4093.264424] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4093.271655] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 4093.276807] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4093.285824] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 4093.795667] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4094.245197] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4094.748507] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 4094.752804] Lustre: Skipped 2 previous similar messages [ 4094.766650] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4094.774591] Lustre: Skipped 2 previous similar messages [ 4094.777488] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 4094.780654] Lustre: Skipped 2 previous similar messages [ 4094.785649] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4094.798996] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 4094.804239] Lustre: Skipped 2 previous similar messages [ 4095.292431] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4095.784889] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4096.230947] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4096.653056] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4097.054102] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 4097.056826] Lustre: Skipped 4 previous similar messages [ 4097.063893] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4097.069483] Lustre: Skipped 4 previous similar messages [ 4097.071895] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 4097.074622] Lustre: Skipped 4 previous similar messages [ 4097.077319] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4097.082830] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 4097.087299] Lustre: Skipped 4 previous similar messages [ 4097.531557] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4097.991573] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4098.430642] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4098.893088] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4099.320921] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4099.736435] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4100.177202] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4100.643807] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4101.075283] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 4101.078327] Lustre: Skipped 8 previous similar messages [ 4101.086745] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4101.094098] Lustre: Skipped 8 previous similar messages [ 4101.098768] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 4101.101641] Lustre: Skipped 8 previous similar messages [ 4101.104145] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4101.114539] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 4101.118158] Lustre: Skipped 8 previous similar messages [ 4101.570236] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4102.068754] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4102.568526] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4103.022992] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4103.493268] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4103.916452] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4104.349847] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4104.848974] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4105.333286] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4105.809722] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4106.297613] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4106.839394] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4107.434455] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4108.002410] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4108.473609] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4108.945653] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4109.440576] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 4109.443669] Lustre: Skipped 16 previous similar messages [ 4109.451489] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4109.459634] Lustre: Skipped 16 previous similar messages [ 4109.461825] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 4109.464921] Lustre: Skipped 16 previous similar messages [ 4109.467151] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4109.474631] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 4109.482241] Lustre: Skipped 16 previous similar messages [ 4109.958147] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4110.504518] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4110.974216] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4111.426855] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4111.956796] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4112.415729] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4112.870268] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4113.308448] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4113.830644] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4114.299919] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4114.765496] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4115.308946] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4115.821587] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4116.336596] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4116.965400] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4117.553240] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4118.113513] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4118.617572] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4119.083885] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4119.561487] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4120.096275] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4120.663538] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4121.178542] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4121.759882] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4122.320457] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4122.848428] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4123.351512] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4123.870264] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4124.347472] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4124.809431] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4125.310700] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4125.774742] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 4125.777563] Lustre: Skipped 31 previous similar messages [ 4125.785956] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4125.792383] Lustre: Skipped 31 previous similar messages [ 4125.795383] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 4125.799872] Lustre: Skipped 31 previous similar messages [ 4125.802961] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4125.810208] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 4125.812842] Lustre: Skipped 31 previous similar messages [ 4126.274697] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4126.750988] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4127.223044] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4127.707637] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4128.195863] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4128.651672] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4129.159241] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4129.667274] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4130.177112] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4130.705360] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4131.306348] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4131.816954] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4132.373446] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4132.884197] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4133.391913] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4133.928826] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4134.425713] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4134.909141] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4135.442223] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4136.026428] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4136.572862] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4137.136362] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4137.770612] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4138.327402] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4138.836727] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4139.341164] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4139.944778] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4140.457153] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4140.975359] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4141.478272] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4141.934952] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4142.394424] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4142.860682] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4143.303910] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4143.766846] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4144.238740] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4144.658593] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4145.117960] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4145.620235] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4146.156133] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4146.613574] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4147.070812] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4147.674716] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4148.204509] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4148.702979] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4149.208648] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4149.709113] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4150.168232] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4150.733477] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4151.265412] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4151.763474] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4152.291954] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4152.844845] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4153.345097] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4153.854627] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4154.296928] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4154.747762] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4155.209136] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4155.692468] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4156.168426] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4156.665578] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4157.167522] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4157.699370] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4158.264824] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 4158.268749] Lustre: Skipped 63 previous similar messages [ 4158.276552] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4158.283490] Lustre: Skipped 63 previous similar messages [ 4158.287735] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 4158.290575] Lustre: Skipped 63 previous similar messages [ 4158.294840] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4158.311199] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 4158.318579] Lustre: Skipped 63 previous similar messages [ 4158.806081] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4159.259745] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4159.711457] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4160.191235] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4160.684075] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4161.230429] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4161.726614] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4162.271836] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4162.875188] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4163.441823] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4163.936679] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4164.424534] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4164.960508] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4165.470869] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4165.950764] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4166.420914] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4166.921246] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4167.371903] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4167.805144] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4168.279845] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4168.772371] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4169.317444] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4169.837360] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4170.328187] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4170.841827] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4171.344667] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4171.833425] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4172.324288] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4172.836119] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4173.322400] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4173.852624] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4174.344561] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4174.808045] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4175.237467] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4175.720357] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4176.221033] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4176.681069] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4177.191598] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4177.720662] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4178.217547] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4178.695827] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4179.175915] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4179.623158] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4180.040525] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4180.533374] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4181.041564] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4181.538159] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4182.000041] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4182.461884] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4182.969502] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4183.443105] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4183.906708] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4184.388496] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4184.810671] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4185.251855] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4185.698305] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4186.161864] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4186.650624] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4187.150212] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4187.619715] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4188.109762] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4188.605291] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4189.065847] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4189.566552] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4190.061263] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4190.546491] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4191.092607] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4191.574889] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4192.079635] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4192.585123] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4193.068682] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4193.553598] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4194.074074] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4194.540347] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4195.011648] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4195.552854] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4196.076442] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4196.570536] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4197.098624] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4197.602530] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4198.066380] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4198.571732] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4199.054949] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4199.495578] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4199.960966] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4200.466468] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4200.929538] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4201.437836] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4201.912301] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4202.308982] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4202.743248] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4203.189886] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4203.630391] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4204.092955] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4204.536658] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4204.974312] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4205.435466] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4205.940839] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4206.487532] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4206.992312] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4207.462068] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4207.921780] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4208.442130] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4208.972080] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4209.474875] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4209.991129] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4210.482485] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4211.045827] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4211.582447] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4212.067314] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4212.574043] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4213.086762] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4213.562068] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4214.083855] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4214.599524] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4215.093168] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4215.603290] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4216.121418] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4216.679122] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4217.185112] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4217.666966] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4218.230688] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4218.751617] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4219.239399] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4219.757640] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4220.263888] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4220.761434] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4221.269453] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4221.796221] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4222.252333] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4222.707542] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 4222.711850] Lustre: Skipped 130 previous similar messages [ 4222.723706] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4222.729953] Lustre: Skipped 130 previous similar messages [ 4222.734755] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 4222.738368] Lustre: Skipped 130 previous similar messages [ 4222.742222] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4222.749897] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 4222.752715] Lustre: Skipped 130 previous similar messages [ 4223.266354] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4223.723973] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4224.240036] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4224.754164] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4225.252234] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4225.734370] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4226.202757] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4226.714839] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4227.234735] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4227.690945] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4228.171105] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4228.623310] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4229.112316] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4229.593877] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4230.099690] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4230.666275] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4231.201482] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4231.817574] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4232.401236] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4232.872641] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4233.393628] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4233.914160] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4234.418495] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4234.896230] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4235.439362] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4235.950665] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4236.468506] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4237.036930] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4237.526504] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4238.056131] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4238.593774] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4239.092830] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4239.596765] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4240.095909] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4240.564463] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4241.125253] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4241.655661] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4242.241036] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4242.838616] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4243.354374] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4243.839186] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4244.367485] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4244.870101] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4245.332792] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4245.806786] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4246.322417] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4246.818904] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4247.338049] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4247.863828] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4248.316734] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4248.760858] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4249.211338] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4249.656566] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4250.092371] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4250.507632] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4250.980419] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4251.471384] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4251.894761] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4252.326781] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4252.800351] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4253.277032] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4253.754714] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4254.217166] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4254.637109] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4255.034229] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4255.464447] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4255.918469] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4256.370463] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4256.836294] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4257.267068] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4257.723400] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4258.201810] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4258.649341] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4259.077819] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4259.494568] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4259.966497] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4260.416673] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4260.894360] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4261.377402] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4261.833850] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4262.287552] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4262.765996] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4263.185435] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4263.637987] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4264.072987] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4264.541177] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4265.046774] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4265.548614] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4266.015098] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4266.480434] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4266.944952] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4267.375332] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4267.831176] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4268.258808] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4268.685662] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4269.106988] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4269.549232] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4270.021264] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4270.502709] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4271.008972] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4271.513663] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4272.005454] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4272.487869] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4272.925853] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4273.356507] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4273.790700] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4274.245535] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4274.680365] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4275.075458] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4275.493722] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4275.866743] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4276.320157] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4276.714838] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4277.129892] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4277.537763] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4277.970200] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4278.373943] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4278.778906] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4279.188237] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4279.603188] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4280.008124] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4280.402615] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4280.843600] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4281.270060] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4281.726695] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4282.138911] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4282.626613] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4283.104179] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4283.555914] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4284.005519] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4284.412250] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4284.793298] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4285.180362] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4285.581382] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4285.956992] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4286.387498] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4286.831775] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4287.308373] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4287.787519] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4288.223813] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4288.660812] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4289.088828] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4289.534529] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4289.953230] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4290.414992] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4290.875542] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4291.281420] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4291.730509] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4292.143467] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4292.553487] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4292.987683] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4293.461232] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4293.930944] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4294.388694] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4294.832856] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4295.327078] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4295.777906] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4296.243457] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4296.772292] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4297.330509] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4297.808099] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4298.300849] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4298.794419] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4299.256860] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4299.800411] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4300.264474] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4300.758215] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4301.300458] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4301.901163] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4302.369785] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4302.842305] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4303.347303] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4303.837389] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4304.295365] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4304.778543] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4305.221860] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4305.687162] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4306.160914] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4306.700838] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4307.208288] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4307.700354] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4308.185293] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4308.705807] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4309.189235] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4309.661854] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4310.139274] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4310.640049] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4311.256257] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4311.737336] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4312.303942] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4312.827311] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4313.342253] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4313.927465] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4314.463574] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4314.963662] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4315.429528] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4315.926355] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4316.493234] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4316.948117] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4317.446611] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4317.903463] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4318.382367] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4318.856723] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4319.308267] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4319.749247] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4320.189574] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4320.619935] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4321.124294] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4321.629364] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4322.156484] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4322.708862] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4323.166973] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4323.683285] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4324.149752] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4324.589649] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4325.050518] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4325.484666] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4325.967501] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4326.423743] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4326.868805] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4327.280503] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4327.684424] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4328.084900] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4328.482532] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4328.890651] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4329.300488] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4329.699304] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4330.122558] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4330.504755] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4330.882429] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4331.265401] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4331.677129] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4332.079189] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4332.478685] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4332.857336] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4333.259251] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4333.662205] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4334.104367] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4334.535989] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4334.952591] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4335.399628] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4335.772141] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4336.168651] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4336.599458] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4337.080058] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4337.527358] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4337.939660] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4338.345908] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4338.775846] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4339.190366] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4339.566247] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4339.945289] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4340.326272] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4340.740430] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4341.163522] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4341.586398] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4342.049453] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4342.473312] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4342.878273] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4343.277237] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4343.748204] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4344.164953] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4344.588531] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4345.027758] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4345.467849] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4345.909840] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4346.318674] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4346.728402] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4347.158391] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4347.563547] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4347.968691] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4348.363681] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4348.775150] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4349.180112] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4349.570526] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4349.974047] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4350.397508] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4350.801039] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 4350.802853] Lustre: Skipped 277 previous similar messages [ 4350.810680] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4350.816861] Lustre: Skipped 277 previous similar messages [ 4350.819552] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 4350.823943] Lustre: Skipped 277 previous similar messages [ 4350.826518] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4350.832471] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 4350.835350] Lustre: Skipped 277 previous similar messages [ 4351.230748] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4351.655074] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4352.086756] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4352.491187] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4352.891529] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4353.291533] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4353.710899] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4354.116637] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4354.516641] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4354.951320] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4355.347020] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4355.726947] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4356.151391] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4356.561062] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4356.963222] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4357.354197] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4357.774937] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4358.200138] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4358.613694] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4359.017216] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4359.417873] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4359.807768] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4360.225145] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4360.624885] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4361.038191] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4361.447143] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4361.847011] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4362.280735] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4362.717856] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4363.157663] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4363.545553] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4363.950076] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4364.364381] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4364.813452] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4365.229478] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4365.668659] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4366.085857] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4366.543528] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4366.980644] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4367.431162] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4367.893662] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4368.338731] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4368.771241] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4369.208374] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4369.610236] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4370.005315] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4370.387145] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4370.781731] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4371.189940] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4371.570357] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4371.932753] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4372.291573] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4372.665943] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4373.040452] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4373.421701] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4373.820243] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4374.191995] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4374.572889] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4374.955184] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4375.356173] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4375.755740] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4376.199523] LustreError: 162704:0:(osp_precreate.c:651:osp_precreate_send()) lustre-OST0000-osc-MDT0000: can't precreate: rc = -5 [ 4376.204818] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4376.205097] LustreError: 162704:0:(osp_precreate.c:1355:osp_precreate_thread()) lustre-OST0000-osc-MDT0000: cannot precreate objects: rc = -5 [ 4376.621848] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4377.046107] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4377.533113] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4377.958550] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4378.371389] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4378.780489] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4379.191578] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4379.644339] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4380.065904] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4380.478080] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4380.910238] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4381.347303] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4381.781145] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4382.233588] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4382.745839] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4383.277391] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4383.723224] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4384.184437] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4384.574972] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4384.983237] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4385.391649] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4385.804941] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4386.231070] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4386.667055] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4387.070917] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4387.470502] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4387.913754] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4388.360587] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4388.805760] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4389.241960] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4389.638694] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4390.028529] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4390.444019] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4390.878079] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4391.304632] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4391.721140] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4392.136051] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4392.563933] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4393.004080] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4393.430555] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4393.832612] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4394.230631] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4394.663592] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4395.118719] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4395.570265] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4396.036344] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4396.535759] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4397.007761] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4397.475618] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4397.941684] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4398.416993] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4398.872272] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4399.317453] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4399.764483] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4400.206179] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4400.607545] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4401.034516] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4401.511150] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4401.959911] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4402.358788] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4402.769101] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4403.157498] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4403.580905] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4403.935650] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4404.331089] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4404.724994] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4405.121458] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4405.567374] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4405.978306] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4406.412689] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4406.850394] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4407.285171] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4407.741574] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4408.149340] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4408.570980] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4408.992341] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4409.432785] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4409.897242] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4410.366596] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4410.805293] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4411.273439] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4411.697593] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4412.128808] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4412.529236] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4412.933387] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4413.354652] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4413.822393] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4414.252139] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4414.694921] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4415.128891] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4415.550590] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4415.958649] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4416.343117] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4416.756136] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4417.188592] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4417.638469] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4418.065135] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4418.482152] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4418.868384] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4419.234556] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4419.604431] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4419.981538] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4420.351979] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4420.714030] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4421.127830] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4421.505042] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4421.906107] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4422.328867] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4422.723474] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4423.092588] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4423.474471] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4423.857472] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4424.252656] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4424.658148] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4425.015570] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4425.404400] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4425.809478] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4426.217193] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4426.577098] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4426.976115] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4427.371455] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4427.786466] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4428.196769] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4428.584866] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4428.989922] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4429.394430] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4429.866646] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4430.323369] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4430.775490] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4431.197738] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4431.609038] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4432.050585] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4432.490652] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4432.933796] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4433.396937] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4433.867129] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4434.344929] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4434.851748] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4435.262610] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4435.673535] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4436.063512] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4436.490146] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4436.951337] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4437.371580] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4437.823774] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4438.261045] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4438.671841] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4439.072673] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4439.478893] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4439.895742] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4440.261515] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4440.640685] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4441.057576] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4441.482922] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4441.941607] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4442.402281] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4442.868441] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4443.326631] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4443.762645] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4444.192237] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4444.654144] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4445.149474] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4445.623651] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4446.167737] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4446.664819] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4447.154514] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4447.649713] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4448.173822] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4448.656762] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4449.126065] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4449.664291] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4450.139416] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4450.596611] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4451.048648] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4451.481756] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4451.892409] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4452.342741] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4452.876763] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4453.314926] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4453.754409] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4454.148903] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4454.567517] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4454.965883] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4455.340555] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4455.729172] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4456.150533] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4456.572354] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4457.011836] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4457.460501] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4457.954581] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4458.416989] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4458.878784] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4459.356775] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4459.847075] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4460.297782] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4460.752939] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4461.235518] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4461.723065] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4462.200665] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4462.737511] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4463.175582] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4463.597657] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4464.063700] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4464.528376] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4465.010248] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4465.496801] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4466.006241] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4466.483156] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4466.892508] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4467.376599] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4467.860377] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4468.298751] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4468.760784] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4469.199453] LustreError: 162704:0:(osp_precreate.c:651:osp_precreate_send()) lustre-OST0000-osc-MDT0000: can't precreate: rc = -5 [ 4469.206366] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4469.206453] LustreError: 162704:0:(osp_precreate.c:1355:osp_precreate_thread()) lustre-OST0000-osc-MDT0000: cannot precreate objects: rc = -5 [ 4469.641982] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4470.054692] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4470.478707] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4505.570987] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 4505.574936] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 4505.577763] Lustre: Skipped 1 previous similar message [ 4510.692298] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 4510.695491] Lustre: Skipped 1 previous similar message [ 4511.219236] LustreError: 207201:0:(obd_class.h:479:obd_check_dev()) Device 21 not setup [ 4511.324275] Lustre: server umount lustre-OST0000 complete [ 4515.296993] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 4515.301151] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 4519.045064] LustreError: 207402:0:(obd_class.h:479:obd_check_dev()) Device 9 not setup [ 4519.047153] LustreError: 207402:0:(obd_class.h:479:obd_check_dev()) Skipped 3 previous similar messages [ 4519.099506] Lustre: server umount lustre-MDT0000 complete [ 4520.630233] LustreError: 207403:0:(ldlm_lockd.c:2607:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1758741506 with bad export cookie 16428467173688422295 [ 4520.632666] LustreError: MGC192.168.201.158@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 4520.634983] LustreError: 207403:0:(ldlm_lockd.c:2607:ldlm_cancel_handler()) Skipped 3 previous similar messages [ 4520.718833] LustreError: 207604:0:(obd_class.h:479:obd_check_dev()) Device 16 not setup [ 4520.721733] LustreError: 207604:0:(obd_class.h:479:obd_check_dev()) Skipped 5 previous similar messages [ 4520.815265] Lustre: server umount lustre-MDT0001 complete [ 4525.629127] Lustre: DEBUG MARKER: oleg158-server.virtnet: executing unload_modules_local [ 4527.097319] Key type lgssc unregistered [ 4527.259311] LNet: 208135:0:(lib-ptl.c:966:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 4527.263285] LNetError: 208135:0:(acceptor.c:263:lnet_acceptor_remove_socket()) Interface ens2 not found [ 4527.274076] LNet: Removed LNI 192.168.201.158@tcp [ 4527.713537] Key type .llcrypt unregistered [ 4527.715151] Key type ._llcrypt unregistered [ 4539.896525] Key type ._llcrypt registered [ 4539.898285] Key type .llcrypt registered [ 4539.950153] Lustre: DEBUG MARKER: oleg158-server.virtnet: executing load_modules_local [ 4540.459147] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 4540.515158] alg: No test for adler32 (adler32-zlib) [ 4541.407788] Lustre: Lustre: Build Version: 2.16.58_53_g2e3f3ed [ 4541.519947] LNet: Added LNI 192.168.201.158@tcp [8/256/0/180] [ 4543.120240] Key type lgssc registered [ 4543.657385] Lustre: Echo OBD driver; http://www.lustre.org/ [ 4547.817550] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 4547.831082] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 4549.094357] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 4551.047035] Lustre: DEBUG MARKER: oleg158-server.virtnet: executing set_default_debug -1 all [ 4555.276842] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 4555.506793] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 4557.420139] Lustre: DEBUG MARKER: oleg158-server.virtnet: executing set_default_debug -1 all [ 4560.021710] Lustre: DEBUG MARKER: oleg158-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 4561.836638] Lustre: DEBUG MARKER: oleg158-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 4565.192946] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 4565.376562] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 4568.138715] Lustre: DEBUG MARKER: oleg158-server.virtnet: executing set_default_debug -1 all [ 4571.094445] Lustre: DEBUG MARKER: oleg158-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 4574.595921] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:9683 to 0x280000401:9729) [ 4579.967362] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 4580.936632] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4580.943071] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 4580.947591] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4580.956078] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 4580.956101] LustreError: 209529:0:(osp_precreate.c:1263:osp_precreate_thread()) cfs_fail_timeout id 2107 sleeping for 20000ms [ 4582.985250] Lustre: lustre-OST0000: Not available for connect from 192.168.201.58@tcp (stopping) [ 4584.930856] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4584.937167] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 4584.939023] Lustre: Skipped 1 previous similar message [ 4588.092769] Lustre: lustre-OST0000: Not available for connect from 192.168.201.58@tcp (stopping) [ 4588.096873] Lustre: Skipped 1 previous similar message [ 4593.209405] Lustre: lustre-OST0000: Not available for connect from 192.168.201.58@tcp (stopping) [ 4593.217480] Lustre: Skipped 2 previous similar messages [ 4596.192404] Lustre: lustre-OST0000 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 4596.221230] LustreError: 211908:0:(obd_class.h:479:obd_check_dev()) Device 21 not setup [ 4596.335408] Lustre: server umount lustre-OST0000 complete [ 4601.040386] LustreError: 209529:0:(osp_precreate.c:1263:osp_precreate_thread()) cfs_fail_timeout id 2107 awake [ 4625.987403] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 4626.150948] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 4628.878144] Lustre: DEBUG MARKER: oleg158-server.virtnet: executing set_default_debug -1 all [ 4631.677643] Lustre: DEBUG MARKER: oleg158-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 4650.670055] LustreError: lustre-OST0000-osc-MDT0001: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4650.675101] Lustre: lustre-OST0000-osc-MDT0001: Connection restored to 0@lo (at 0@lo) [ 4650.675234] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4650.694670] Lustre: DEBUG MARKER: osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid in FULL state after 17 sec [ 4652.837374] Lustre: DEBUG MARKER: oleg158-client.virtnet: executing wait_import_state (FULL|IDLE) osc.lustre-OST0000-osc-ffff946ec9ea9800.ost_server_uuid 50 [ 4653.892764] Lustre: DEBUG MARKER: osc.lustre-OST0000-osc-ffff946ec9ea9800.ost_server_uuid in FULL state after 0 sec [ 4661.220375] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 4661.220694] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4661.222408] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 4661.222414] Lustre: Skipped 2 previous similar messages [ 4661.238067] Lustre: Skipped 1 previous similar message [ 4662.268185] LustreError: 213250:0:(obd_class.h:479:obd_check_dev()) Device 21 not setup [ 4662.277872] LustreError: 213250:0:(obd_class.h:479:obd_check_dev()) Skipped 3 previous similar messages [ 4662.361431] Lustre: server umount lustre-OST0000 complete [ 4666.338635] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4666.344670] Lustre: Skipped 1 previous similar message [ 4670.480551] LustreError: 213452:0:(obd_class.h:479:obd_check_dev()) Device 9 not setup [ 4670.483233] LustreError: 213452:0:(obd_class.h:479:obd_check_dev()) Skipped 3 previous similar messages [ 4670.520680] Lustre: server umount lustre-MDT0000 complete [ 4671.458861] LustreError: 211661:0:(ldlm_lib.c:1132:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 4671.470748] LustreError: 211661:0:(ldlm_lib.c:1132:target_handle_connect()) Skipped 3 previous similar messages [ 4672.694492] LustreError: 209490:0:(ldlm_lockd.c:2607:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1758741658 with bad export cookie 16581027331654175048 [ 4672.698052] LustreError: MGC192.168.201.158@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 4672.700859] LustreError: 209490:0:(ldlm_lockd.c:2607:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 4672.797327] LustreError: 213654:0:(obd_class.h:479:obd_check_dev()) Device 16 not setup [ 4672.800782] LustreError: 213654:0:(obd_class.h:479:obd_check_dev()) Skipped 5 previous similar messages [ 4672.918355] Lustre: server umount lustre-MDT0001 complete [ 4678.600325] Lustre: DEBUG MARKER: oleg158-server.virtnet: executing unload_modules_local [ 4680.077651] Key type lgssc unregistered [ 4680.242707] LNet: 214186:0:(lib-ptl.c:966:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 4680.248547] LNetError: 214186:0:(acceptor.c:263:lnet_acceptor_remove_socket()) Interface ens2 not found [ 4680.263429] LNet: Removed LNI 192.168.201.158@tcp [ 4680.686225] Key type .llcrypt unregistered [ 4680.687857] Key type ._llcrypt unregistered [ 4698.923234] Key type ._llcrypt registered [ 4698.924976] Key type .llcrypt registered [ 4698.973497] Lustre: DEBUG MARKER: oleg158-server.virtnet: executing load_modules_local [ 4699.455735] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 4699.486701] alg: No test for adler32 (adler32-zlib) [ 4700.379401] Lustre: Lustre: Build Version: 2.16.58_53_g2e3f3ed [ 4700.496728] LNet: Added LNI 192.168.201.158@tcp [8/256/0/180] [ 4702.104192] Key type lgssc registered [ 4702.689648] Lustre: Echo OBD driver; http://www.lustre.org/ [ 4707.388989] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 4707.400476] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 4708.626873] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 4710.221132] Lustre: DEBUG MARKER: oleg158-server.virtnet: executing set_default_debug -1 all [ 4713.690069] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 4713.857071] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 4715.426434] Lustre: DEBUG MARKER: oleg158-server.virtnet: executing set_default_debug -1 all [ 4716.650837] Lustre: Modifying parameter general.debug_raw_pointers=Y in log params [ 4719.504842] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 4719.662730] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 4721.927684] Lustre: DEBUG MARKER: oleg158-server.virtnet: executing set_default_debug -1 all [ 4725.559482] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 4725.591541] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 4725.665905] Lustre: lustre-OST0001: new disk, initializing [ 4725.669560] Lustre: srv-lustre-OST0001: No data found on store. Initialize space. [ 4725.711331] Lustre: lustre-OST0001: Imperative Recovery not enabled, recovery window 60-180 [ 4727.817925] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x00000002c0000400-0x0000000300000400]:1:ost [ 4727.822842] Lustre: cli-lustre-OST0001-super: Allocated super-sequence [0x00000002c0000400-0x0000000300000400]:1:ost] [ 4727.874050] Lustre: lustre-OST0001-osc-MDT0000: update sequence from 0x100010000 to 0x2c0000401 [ 4727.992373] Lustre: DEBUG MARKER: oleg158-server.virtnet: executing set_default_debug -1 all [ 4732.966042] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:9731 to 0x280000401:9761) [ 4734.144351] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 4740.268892] Lustre: Setting parameter general.lod.*.mdt_hash=crush in log params [ 4754.913893] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 4754.917455] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4754.924190] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 4758.499317] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 4758.500480] Lustre: lustre-MDT0000-lwp-OST0000: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4758.503047] Lustre: Skipped 2 previous similar messages [ 4758.507389] Lustre: Skipped 2 previous similar messages [ 4760.330349] LustreError: 219230:0:(obd_class.h:479:obd_check_dev()) Device 28 not setup [ 4760.372386] Lustre: server umount lustre-MDT0000 complete [ 4761.890299] LustreError: 218892:0:(ldlm_lockd.c:2607:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1758741748 with bad export cookie 13602228073298839332 [ 4761.893153] LustreError: MGC192.168.201.158@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 4761.895802] LustreError: 218892:0:(ldlm_lockd.c:2607:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 4761.968074] LustreError: 219431:0:(obd_class.h:479:obd_check_dev()) Device 27 not setup [ 4761.971225] LustreError: 219431:0:(obd_class.h:479:obd_check_dev()) Skipped 7 previous similar messages [ 4762.079853] Lustre: server umount lustre-MDT0001 complete [ 4772.852200] LustreError: 219631:0:(obd_class.h:479:obd_check_dev()) Device 21 not setup [ 4772.855490] LustreError: 219631:0:(obd_class.h:479:obd_check_dev()) Skipped 7 previous similar messages [ 4772.875722] Lustre: server umount lustre-OST0000 complete [ 4784.563167] LustreError: 219833:0:(obd_class.h:479:obd_check_dev()) Device 2 not setup [ 4784.566528] LustreError: 219833:0:(obd_class.h:479:obd_check_dev()) Skipped 3 previous similar messages [ 4784.654606] Lustre: server umount lustre-OST0001 complete [ 4786.134525] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro [ 4786.181753] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 4786.722239] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro [ 4786.769263] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 4787.263208] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: ,errors=remount-ro [ 4787.322675] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 4787.814993] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: ,errors=remount-ro [ 4787.855984] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 4793.981746] Lustre: DEBUG MARKER: oleg158-server.virtnet: executing load_modules_local [ 4798.351757] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 4798.579336] LustreError: 221241:0:(ldlm_lib.c:1132:target_handle_connect()) mylustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 4798.589840] LustreError: 221241:0:(ldlm_lib.c:1132:target_handle_connect()) Skipped 2 previous similar messages [ 4798.623775] Lustre: mylustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 4800.329179] Lustre: DEBUG MARKER: oleg158-server.virtnet: executing set_default_debug -1 all [ 4804.066779] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 4804.067659] LustreError: 221241:0:(ldlm_lib.c:1132:target_handle_connect()) mylustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 4806.004680] Lustre: DEBUG MARKER: oleg158-server.virtnet: executing set_default_debug -1 all [ 4807.254511] Lustre: 222353:0:(mgs_llog.c:1345:mgs_modify_param()) MGS: modify general/debug_raw_pointers=Y (mode = 0) failed: rc = -17 [ 4810.261609] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 4810.428131] Lustre: mylustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 4810.431849] Lustre: Skipped 1 previous similar message [ 4812.913914] Lustre: DEBUG MARKER: oleg158-server.virtnet: executing set_default_debug -1 all [ 4814.504264] LustreError: 223220:0:(ldlm_lib.c:1132:target_handle_connect()) mylustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 4816.847042] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 4819.411912] Lustre: DEBUG MARKER: oleg158-server.virtnet: executing set_default_debug -1 all [ 4823.214429] Lustre: mylustre-OST0001: new connection from mylustre-MDT0000-mdtlov (cleaning up unused objects from 0x2c0000401:3 to 0x2c0000401:33) [ 4823.224686] Lustre: mylustre-OST0000: new connection from mylustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:9763 to 0x280000401:9793) [ 4828.375404] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 4829.873516] Lustre: 224198:0:(mgs_llog.c:1345:mgs_modify_param()) MGS: modify general/lod.*.mdt_hash=crush (mode = 0) failed: rc = -17 [ 4840.417456] LustreError: mylustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 4840.420502] Lustre: mylustre-MDT0000-osp-MDT0001: Connection to mylustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4840.426863] Lustre: mylustre-MDT0000: Not available for connect from 0@lo (stopping) [ 4840.429512] Lustre: Skipped 1 previous similar message [ 4844.004085] Lustre: mylustre-MDT0000-lwp-MDT0001: Connection to mylustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4844.004751] Lustre: mylustre-MDT0000: Not available for connect from 0@lo (stopping) [ 4844.009488] Lustre: Skipped 1 previous similar message [ 4844.014523] Lustre: Skipped 3 previous similar messages [ 4845.823376] LustreError: 224504:0:(obd_class.h:479:obd_check_dev()) Device 10 not setup [ 4845.825419] LustreError: 224504:0:(obd_class.h:479:obd_check_dev()) Skipped 5 previous similar messages [ 4845.856958] Lustre: server umount mylustre-MDT0000 complete [ 4847.009412] LustreError: 221240:0:(ldlm_lib.c:1132:target_handle_connect()) mylustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 4847.016870] LustreError: 221240:0:(ldlm_lib.c:1132:target_handle_connect()) Skipped 3 previous similar messages [ 4847.238516] LustreError: 221217:0:(ldlm_lockd.c:2607:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1758741833 with bad export cookie 13602228073298842125 [ 4847.242051] LustreError: MGC192.168.201.158@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 4847.244174] LustreError: 221217:0:(ldlm_lockd.c:2607:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 4847.402293] Lustre: server umount mylustre-MDT0001 complete [ 4859.381629] LustreError: 224906:0:(obd_class.h:479:obd_check_dev()) Device 23 not setup [ 4859.384145] LustreError: 224906:0:(obd_class.h:479:obd_check_dev()) Skipped 15 previous similar messages [ 4859.408095] Lustre: server umount mylustre-OST0000 complete [ 4871.299990] Lustre: server umount mylustre-OST0001 complete [ 4872.549510] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro [ 4872.584027] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 4873.023637] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro [ 4873.063025] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 4873.524031] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: ,errors=remount-ro [ 4873.566302] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 4874.028974] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: ,errors=remount-ro [ 4874.075706] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 4880.047444] Lustre: DEBUG MARKER: oleg158-server.virtnet: executing load_modules_local [ 4884.400932] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 4884.618920] LustreError: 226516:0:(ldlm_lib.c:1132:target_handle_connect()) tfs-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 4884.655046] Lustre: tfs-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 4884.657728] Lustre: Skipped 1 previous similar message [ 4886.334470] Lustre: DEBUG MARKER: oleg158-server.virtnet: executing set_default_debug -1 all [ 4890.039164] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 4891.848514] Lustre: DEBUG MARKER: oleg158-server.virtnet: executing set_default_debug -1 all [ 4893.008829] Lustre: 227625:0:(mgs_llog.c:1345:mgs_modify_param()) MGS: modify general/debug_raw_pointers=Y (mode = 0) failed: rc = -17 [ 4895.511224] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 4897.863584] Lustre: DEBUG MARKER: oleg158-server.virtnet: executing set_default_debug -1 all [ 4900.773134] LustreError: 227984:0:(ldlm_lib.c:1132:target_handle_connect()) tfs-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 4900.781100] LustreError: 227984:0:(ldlm_lib.c:1132:target_handle_connect()) Skipped 1 previous similar message [ 4901.308248] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 4903.701673] Lustre: DEBUG MARKER: oleg158-server.virtnet: executing set_default_debug -1 all [ 4909.545568] Lustre: tfs-OST0000: new connection from tfs-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:9795 to 0x280000401:9825) [ 4909.547702] Lustre: tfs-OST0001: new connection from tfs-MDT0000-mdtlov (cleaning up unused objects from 0x2c0000401:3 to 0x2c0000401:65) [ 4912.926459] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 4914.519264] Lustre: 229474:0:(mgs_llog.c:1345:mgs_modify_param()) MGS: modify general/lod.*.mdt_hash=crush (mode = 0) failed: rc = -17 [ 4924.899025] Lustre: tfs-MDT0000-osp-MDT0001: Connection to tfs-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4924.902894] Lustre: tfs-MDT0000: Not available for connect from 0@lo (stopping) [ 4924.903914] Lustre: Skipped 3 previous similar messages [ 4924.906647] Lustre: Skipped 1 previous similar message [ 4928.194895] LustreError: 229722:0:(obd_class.h:479:obd_check_dev()) Device 10 not setup [ 4928.197132] LustreError: 229722:0:(obd_class.h:479:obd_check_dev()) Skipped 9 previous similar messages [ 4928.232593] Lustre: server umount tfs-MDT0000 complete [ 4929.798479] LustreError: 227243:0:(ldlm_lockd.c:2607:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1758741916 with bad export cookie 13602228073298844015 [ 4929.802380] LustreError: MGC192.168.201.158@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 4929.804541] LustreError: 227243:0:(ldlm_lockd.c:2607:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 4929.976466] Lustre: server umount tfs-MDT0001 complete [ 4941.763868] Lustre: server umount tfs-OST0000 complete [ 4953.464866] Lustre: server umount tfs-OST0001 complete [ 4954.682427] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro [ 4954.714552] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 4955.139412] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro [ 4955.173296] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 4955.585025] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: ,errors=remount-ro [ 4955.616877] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 4956.052281] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: ,errors=remount-ro [ 4956.078936] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 4961.884459] Lustre: DEBUG MARKER: oleg158-server.virtnet: executing load_modules_local [ 4965.943705] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 4966.170322] LustreError: 231736:0:(ldlm_lib.c:1132:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 4966.208070] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 4966.211431] Lustre: Skipped 3 previous similar messages [ 4967.840079] Lustre: DEBUG MARKER: oleg158-server.virtnet: executing set_default_debug -1 all [ 4971.548740] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 4973.380559] Lustre: DEBUG MARKER: oleg158-server.virtnet: executing set_default_debug -1 all [ 4974.552560] Lustre: 232846:0:(mgs_llog.c:1345:mgs_modify_param()) MGS: modify general/debug_raw_pointers=Y (mode = 0) failed: rc = -17 [ 4977.234540] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 4979.581223] Lustre: DEBUG MARKER: oleg158-server.virtnet: executing set_default_debug -1 all [ 4982.706643] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 4984.864649] Lustre: DEBUG MARKER: oleg158-server.virtnet: executing set_default_debug -1 all [ 4991.466111] Lustre: lustre-OST0001: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x2c0000401:3 to 0x2c0000401:97) [ 4991.466381] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:9795 to 0x280000401:9857) [ 4999.022663] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 5000.528301] Lustre: 234714:0:(mgs_llog.c:1345:mgs_modify_param()) MGS: modify general/lod.*.mdt_hash=crush (mode = 0) failed: rc = -17 [ 5008.048846] Lustre: DEBUG MARKER: == conf-sanity test 104a: Make sure user defined options are reflected in mount ========================================================== 15:26:34 (1758741994) [ 5022.179549] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5022.183356] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 5022.184781] Lustre: Skipped 3 previous similar messages [ 5022.189993] Lustre: Skipped 3 previous similar messages [ 5026.117509] LustreError: 235083:0:(obd_class.h:479:obd_check_dev()) Device 10 not setup [ 5026.120188] LustreError: 235083:0:(obd_class.h:479:obd_check_dev()) Skipped 25 previous similar messages [ 5026.149510] Lustre: server umount lustre-MDT0000 complete [ 5027.298071] LustreError: 231735:0:(ldlm_lib.c:1132:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 5027.305110] LustreError: 231735:0:(ldlm_lib.c:1132:target_handle_connect()) Skipped 5 previous similar messages [ 5027.575269] LustreError: 231712:0:(ldlm_lockd.c:2607:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1758742013 with bad export cookie 13602228073298845681 [ 5027.579230] LustreError: MGC192.168.201.158@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 5027.580359] LustreError: 231712:0:(ldlm_lockd.c:2607:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 5033.814790] Lustre: server umount lustre-MDT0001 complete [ 5041.211199] Lustre: server umount lustre-OST0000 complete [ 5048.571476] Lustre: server umount lustre-OST0001 complete [ 5050.776735] Lustre: DEBUG MARKER: oleg158-server.virtnet: executing set_hostid [ 5054.406338] Lustre: DEBUG MARKER: oleg158-server.virtnet: executing load_modules_local [ 5058.284555] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 5060.771070] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 5062.970638] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 5065.375561] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 5068.096567] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 5068.118809] LDISKFS-fs (dm-0): Mount option "noacl" will be removed by 3.5 [ 5068.118809] Contact linux-ldiskfs@vger.kernel.org if you think we should keep it. [ 5068.118809] [ 5068.126209] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: acl,user_xattr,errors=remount-ro,noacl,no_mbcache,nodelalloc [ 5068.225763] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 5068.239148] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 5068.278793] Lustre: lustre-MDT0000: new disk, initializing [ 5068.309631] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 5068.313183] Lustre: Skipped 3 previous similar messages [ 5068.320766] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 5069.801595] Lustre: DEBUG MARKER: oleg158-server.virtnet: executing set_default_debug -1 all [ 5074.382377] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 5074.411694] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 5074.453643] Lustre: 237868:0:(mgs_llog.c:1345:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 5074.481177] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [ 5074.484036] Lustre: Skipped 1 previous similar message [ 5074.523755] Lustre: lustre-MDT0001: new disk, initializing [ 5074.580057] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [ 5074.584906] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 5076.089839] Lustre: DEBUG MARKER: oleg158-server.virtnet: executing set_default_debug -1 all [ 5080.529712] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 5080.555663] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 5080.660631] Lustre: lustre-OST0000: new disk, initializing [ 5080.663155] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 5082.045302] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [ 5082.049827] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 5082.088763] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 5082.769549] Lustre: DEBUG MARKER: oleg158-server.virtnet: executing set_default_debug -1 all [ 5087.374943] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 5087.402501] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 5087.469662] Lustre: lustre-OST0001: new disk, initializing [ 5087.472410] Lustre: srv-lustre-OST0001: No data found on store. Initialize space. [ 5089.348679] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x00000002c0000400-0x0000000300000400]:1:ost [ 5089.351852] Lustre: cli-lustre-OST0001-super: Allocated super-sequence [0x00000002c0000400-0x0000000300000400]:1:ost] [ 5089.401450] Lustre: lustre-OST0001-osc-MDT0000: update sequence from 0x100010000 to 0x2c0000401 [ 5089.664604] Lustre: DEBUG MARKER: oleg158-server.virtnet: executing set_default_debug -1 all [ 5094.431143] Lustre: DEBUG MARKER: == conf-sanity test 104b: Mount uses last flock argument ========================================================== 15:28:01 (1758742081) [ 5098.605705] Lustre: DEBUG MARKER: == conf-sanity test 105: check file creation for ro and rw bind mnt pt ========================================================== 15:28:05 (1758742085) [ 5112.801161] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 5112.801199] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5112.803547] LustreError: Skipped 1 previous similar message [ 5112.807626] Lustre: Skipped 3 previous similar messages [ 5112.809625] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 5112.812945] Lustre: Skipped 3 previous similar messages [ 5116.207134] LustreError: 241292:0:(obd_class.h:479:obd_check_dev()) Device 19 not setup [ 5116.209953] LustreError: 241292:0:(obd_class.h:479:obd_check_dev()) Skipped 25 previous similar messages [ 5116.231946] Lustre: server umount lustre-OST0000 complete [ 5119.971253] LustreError: 239405:0:(ldlm_lib.c:1132:target_handle_connect()) lustre-OST0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 5119.977421] LustreError: 239405:0:(ldlm_lib.c:1132:target_handle_connect()) Skipped 2 previous similar messages [ 5123.803867] Lustre: server umount lustre-MDT0000 complete [ 5125.302898] LustreError: 237904:0:(ldlm_lockd.c:2607:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1758742111 with bad export cookie 13602228073298847235 [ 5125.308126] LustreError: MGC192.168.201.158@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 5125.308638] LustreError: 237904:0:(ldlm_lockd.c:2607:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 5131.556091] Lustre: server umount lustre-MDT0001 complete [ 5135.623042] Lustre: DEBUG MARKER: oleg158-server.virtnet: executing unload_modules_local [ 5154.272170] Lustre: lustre-OST0001 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 5154.366743] Lustre: server umount lustre-OST0001 complete [ 5156.679810] Lustre: DEBUG MARKER: oleg158-server.virtnet: executing set_hostid [ 5163.220627] Lustre: DEBUG MARKER: oleg158-server.virtnet: executing load_modules_local [ 5166.955247] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 5169.478111] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 5171.699211] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 5174.086267] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 5176.614097] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 5176.645972] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 5176.745808] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 5176.757420] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 5176.794165] Lustre: lustre-MDT0000: new disk, initializing [ 5176.834896] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 5178.329960] Lustre: DEBUG MARKER: oleg158-server.virtnet: executing set_default_debug -1 all [ 5183.144157] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 5183.178387] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 5183.223053] Lustre: 244751:0:(mgs_llog.c:1345:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 5183.355899] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 5184.928699] Lustre: DEBUG MARKER: oleg158-server.virtnet: executing set_default_debug -1 all [ 5188.422811] Lustre: DEBUG MARKER: oleg158-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 5189.793409] Lustre: DEBUG MARKER: oleg158-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 5192.422787] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 5192.446680] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 5193.670705] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [ 5193.674283] Lustre: Skipped 1 previous similar message [ 5193.676274] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 5193.715958] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 5194.664843] Lustre: DEBUG MARKER: oleg158-server.virtnet: executing set_default_debug -1 all [ 5198.094807] Lustre: DEBUG MARKER: oleg158-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 5203.938734] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 5203.940543] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5203.943218] LustreError: Skipped 1 previous similar message [ 5203.949102] Lustre: Skipped 4 previous similar messages [ 5203.952016] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 5203.955307] Lustre: Skipped 7 previous similar messages [ 5206.771168] Lustre: server umount lustre-OST0000 complete [ 5209.058420] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 5214.295915] Lustre: server umount lustre-MDT0000 complete [ 5215.598161] LustreError: 244744:0:(ldlm_lockd.c:2607:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1758742201 with bad export cookie 13602228073298848705 [ 5215.602036] LustreError: MGC192.168.201.158@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 5215.603412] LustreError: 244744:0:(ldlm_lockd.c:2607:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 5215.744483] Lustre: server umount lustre-MDT0001 complete [ 5219.583829] Lustre: DEBUG MARKER: oleg158-server.virtnet: executing unload_modules_local [ 5220.767836] Key type lgssc unregistered [ 5220.901642] LNet: 248127:0:(lib-ptl.c:966:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 5220.905320] LNetError: 248127:0:(acceptor.c:263:lnet_acceptor_remove_socket()) Interface ens2 not found [ 5220.915552] LNet: Removed LNI 192.168.201.158@tcp [ 5221.260118] Key type .llcrypt unregistered [ 5221.261700] Key type ._llcrypt unregistered [ 5231.460265] Key type ._llcrypt registered [ 5231.461388] Key type .llcrypt registered [ 5231.497839] Lustre: DEBUG MARKER: oleg158-server.virtnet: executing load_modules_local [ 5231.876662] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 5231.886921] alg: No test for adler32 (adler32-zlib) [ 5232.755468] Lustre: Lustre: Build Version: 2.16.58_53_g2e3f3ed [ 5232.852928] LNet: Added LNI 192.168.201.158@tcp [8/256/0/180] [ 5234.440232] Key type lgssc registered [ 5234.829416] Lustre: Echo OBD driver; http://www.lustre.org/ [ 5237.502071] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 5237.508082] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 5238.670207] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 5239.960586] Lustre: DEBUG MARKER: oleg158-server.virtnet: executing set_default_debug -1 all [ 5242.869112] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 5242.993738] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 5244.352548] Lustre: DEBUG MARKER: oleg158-server.virtnet: executing set_default_debug -1 all [ 5246.345264] Lustre: DEBUG MARKER: oleg158-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 5247.677854] Lustre: DEBUG MARKER: oleg158-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 5250.138457] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 5250.263649] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 5252.384857] Lustre: DEBUG MARKER: oleg158-server.virtnet: executing set_default_debug -1 all [ 5254.389222] Lustre: DEBUG MARKER: oleg158-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 5254.755560] Lustre: Setting parameter lustre-OST0000.ost.unknown_param=50 in log lustre-OST0000 [ 5258.722229] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5258.727992] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 5258.728501] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 5261.483124] LustreError: 251841:0:(obd_class.h:479:obd_check_dev()) Device 21 not setup [ 5261.543853] Lustre: server umount lustre-OST0000 complete [ 5263.329043] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 5263.332429] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5263.338518] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 5267.937549] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 5267.937776] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5273.057483] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 5273.060270] Lustre: Skipped 2 previous similar messages [ 5277.152126] Lustre: lustre-MDT0000 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 5277.186529] LustreError: 252043:0:(obd_class.h:479:obd_check_dev()) Device 9 not setup [ 5277.189264] LustreError: 252043:0:(obd_class.h:479:obd_check_dev()) Skipped 3 previous similar messages [ 5277.212690] Lustre: server umount lustre-MDT0000 complete [ 5278.177611] LustreError: 249547:0:(ldlm_lib.c:1132:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 5278.183190] LustreError: 249547:0:(ldlm_lib.c:1132:target_handle_connect()) Skipped 3 previous similar messages [ 5278.466313] LustreError: 251165:0:(ldlm_lockd.c:2607:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1758742264 with bad export cookie 4124270225594174852 [ 5278.468776] LustreError: MGC192.168.201.158@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 5278.470842] LustreError: 251165:0:(ldlm_lockd.c:2607:ldlm_cancel_handler()) Skipped 3 previous similar messages [ 5278.540127] LustreError: 252249:0:(obd_class.h:479:obd_check_dev()) Device 16 not setup [ 5278.542285] LustreError: 252249:0:(obd_class.h:479:obd_check_dev()) Skipped 5 previous similar messages [ 5278.623941] Lustre: server umount lustre-MDT0001 complete [ 5282.484875] Lustre: DEBUG MARKER: oleg158-server.virtnet: executing unload_modules_local [ 5283.578815] Key type lgssc unregistered [ 5283.720419] LNet: 252780:0:(lib-ptl.c:966:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 5283.723507] LNetError: 252780:0:(acceptor.c:263:lnet_acceptor_remove_socket()) Interface ens2 not found [ 5283.731416] LNet: Removed LNI 192.168.201.158@tcp [ 5284.053059] Key type .llcrypt unregistered [ 5284.054230] Key type ._llcrypt unregistered [ 5290.830513] Key type ._llcrypt registered [ 5290.831495] Key type .llcrypt registered [ 5290.868788] Lustre: DEBUG MARKER: oleg158-server.virtnet: executing load_modules_local [ 5291.299401] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 5291.313114] alg: No test for adler32 (adler32-zlib) [ 5292.168474] Lustre: Lustre: Build Version: 2.16.58_53_g2e3f3ed [ 5292.254884] LNet: Added LNI 192.168.201.158@tcp [8/256/0/180] [ 5293.840192] Key type lgssc registered [ 5294.234716] Lustre: Echo OBD driver; http://www.lustre.org/ [ 5297.440660] Lustre: lustre-OST0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 5297.447748] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 5314.912339] LustreError: Failed to get MGS log lustre-sptlrpc and no local copy. [ 5320.032357] LustreError: 253932:0:(mgc_request.c:233:do_config_log_add()) MGC192.168.201.158@tcp: failed processing log, type 4: rc = -110 [ 5350.752201] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 5352.863094] Lustre: DEBUG MARKER: oleg158-server.virtnet: executing set_default_debug -1 all [ 5354.884448] Lustre: DEBUG MARKER: oleg158-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 5355.695305] LustreError: 254596:0:(obd_class.h:479:obd_check_dev()) Device 1 not setup [ 5355.769308] Lustre: server umount lustre-OST0000 complete [ 5361.473291] Lustre: DEBUG MARKER: oleg158-server.virtnet: executing unload_modules_local [ 5362.586118] Key type lgssc unregistered [ 5362.716368] LNet: 255431:0:(lib-ptl.c:966:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 5362.720417] LNetError: 255431:0:(acceptor.c:263:lnet_acceptor_remove_socket()) Interface ens2 not found [ 5362.728299] LNet: Removed LNI 192.168.201.158@tcp [ 5363.015420] Key type .llcrypt unregistered [ 5363.016702] Key type ._llcrypt unregistered [ 5376.556919] Key type ._llcrypt registered [ 5376.557846] Key type .llcrypt registered [ 5376.590821] Lustre: DEBUG MARKER: oleg158-server.virtnet: executing load_modules_local [ 5376.891074] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 5376.939644] alg: No test for adler32 (adler32-zlib) [ 5377.782324] Lustre: Lustre: Build Version: 2.16.58_53_g2e3f3ed [ 5377.854723] LNet: Added LNI 192.168.201.158@tcp [8/256/0/180] [ 5379.432181] Key type lgssc registered [ 5379.857725] Lustre: Echo OBD driver; http://www.lustre.org/ [ 5384.595767] loop0: detected capacity change from 0 to 204800000 [ 5384.661172] blk_update_request: operation not supported error, dev loop0, sector 399872 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [ 5384.664383] blk_update_request: operation not supported error, dev loop0, sector 208 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [ 5384.796731] blk_update_request: operation not supported error, dev loop0, sector 160232 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [ 5384.801727] blk_update_request: operation not supported error, dev loop0, sector 160240 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [ 5384.824552] blk_update_request: operation not supported error, dev loop0, sector 160344 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [ 5384.844784] blk_update_request: operation not supported error, dev loop0, sector 168552 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [ 5384.861370] blk_update_request: operation not supported error, dev loop0, sector 173280 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [ 5384.885216] blk_update_request: operation not supported error, dev loop0, sector 176912 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [ 5384.913679] blk_update_request: operation not supported error, dev loop0, sector 185112 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [ 5384.944549] LDISKFS-fs (loop0): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 5385.313693] loop0: detected capacity change from 0 to 204800000 [ 5385.395565] blk_update_request: operation not supported error, dev loop0, sector 399872 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [ 5385.684049] LDISKFS-fs (loop0): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 5386.077949] loop0: detected capacity change from 0 to 204800000 [ 5386.242980] LDISKFS-fs (loop0): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 5386.645120] loop0: detected capacity change from 0 to 204800000 [ 5386.850832] LDISKFS-fs (loop0): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 5387.194935] loop0: detected capacity change from 0 to 204800000 [ 5387.208868] LDISKFS-fs (loop0): mounted filesystem with ordered data mode. Opts: (null) [ 5388.633716] loop0: detected capacity change from 0 to 204800000 [ 5388.647459] LDISKFS-fs (loop0): mounted filesystem with ordered data mode. Opts: (null) [ 5390.098373] loop0: detected capacity change from 0 to 204800000 [ 5390.109380] LDISKFS-fs (loop0): mounted filesystem with ordered data mode. Opts: (null) [ 5391.673324] loop0: detected capacity change from 0 to 204800000 [ 5391.682526] LDISKFS-fs (loop0): mounted filesystem with ordered data mode. Opts: (null) [ 5393.006526] loop0: detected capacity change from 0 to 204800000 [ 5393.043850] LDISKFS-fs (loop0): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 5393.073802] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 5393.080541] LDISKFS-fs (loop0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 5393.101420] Lustre: 258806:0:(osd_scrub.c:1842:osd_index_restore()) lustre-MDT0000: restore index 'fld' with [0x200000001:0x3:0x0]: rc = 0 [ 5393.104978] Lustre: 258806:0:(osd_scrub.c:1842:osd_index_restore()) lustre-MDT0000: restore index '0x1010000' with [0x200000003:0xb:0x0]: rc = 0 [ 5393.108069] Lustre: 258806:0:(osd_scrub.c:1842:osd_index_restore()) lustre-MDT0000: restore index '0x1010000-MDT0000' with [0x200000005:0x2:0x0]: rc = 0 [ 5393.112050] Lustre: 258806:0:(osd_scrub.c:1842:osd_index_restore()) lustre-MDT0000: restore index '0x2010000' with [0x200000003:0xc:0x0]: rc = 0 [ 5393.115567] Lustre: 258806:0:(osd_scrub.c:1842:osd_index_restore()) lustre-MDT0000: restore index '0x10000-MDT0000' with [0x200000005:0x1:0x0]: rc = 0 [ 5393.118758] Lustre: 258806:0:(osd_scrub.c:1842:osd_index_restore()) lustre-MDT0000: restore index '0x2010000-MDT0000' with [0x200000005:0x3:0x0]: rc = 0 [ 5393.121853] Lustre: 258806:0:(osd_scrub.c:1842:osd_index_restore()) lustre-MDT0000: restore index '0x10000' with [0x200000003:0xa:0x0]: rc = 0 [ 5393.125958] Lustre: 258806:0:(osd_scrub.c:1842:osd_index_restore()) lustre-MDT0000: restore index 'nodemap' with [0x200000003:0x3:0x0]: rc = 0 [ 5393.128956] Lustre: 258806:0:(osd_scrub.c:1842:osd_index_restore()) lustre-MDT0000: restore index '0x1010000' with [0x200000006:0x1010000:0x0]: rc = 0 [ 5393.133481] Lustre: 258806:0:(osd_scrub.c:1842:osd_index_restore()) lustre-MDT0000: restore index '0x2010000' with [0x200000006:0x2010000:0x0]: rc = 0 [ 5393.137829] Lustre: 258806:0:(osd_scrub.c:1842:osd_index_restore()) lustre-MDT0000: restore index '0x10000' with [0x200000006:0x10000:0x0]: rc = 0 [ 5393.141229] Lustre: 258806:0:(osd_scrub.c:1842:osd_index_restore()) lustre-MDT0000: restore index '0x20000' with [0x200000006:0x20000:0x0]: rc = 0 [ 5393.145395] Lustre: 258806:0:(osd_scrub.c:1842:osd_index_restore()) lustre-MDT0000: restore index '0x2020000' with [0x200000006:0x2020000:0x0]: rc = 0 [ 5393.148330] Lustre: 258806:0:(osd_scrub.c:1842:osd_index_restore()) lustre-MDT0000: restore index '0x1020000' with [0x200000006:0x1020000:0x0]: rc = 0 [ 5395.688987] LustreError: 259072:0:(obd_class.h:479:obd_check_dev()) Device 2 not setup [ 5395.733212] Lustre: server umount MGS complete [ 5396.027094] loop0: detected capacity change from 0 to 204800000 [ 5396.057095] LDISKFS-fs (loop0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 5396.221347] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 5396.241659] Lustre: lustre-MDT0000: Aborting client recovery [ 5396.573113] loop1: detected capacity change from 0 to 204800000 [ 5396.640433] LDISKFS-fs (loop1): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 5396.673027] LDISKFS-fs (loop1): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 5396.694329] Lustre: 259241:0:(osd_scrub.c:1842:osd_index_restore()) lustre-MDT0001: restore index 'fld' with [0x200000001:0x3:0x0]: rc = 0 [ 5396.698325] Lustre: 259241:0:(osd_scrub.c:1842:osd_index_restore()) lustre-MDT0001: restore index '0x10000' with [0x200000003:0x4:0x0]: rc = 0 [ 5396.702157] Lustre: 259241:0:(osd_scrub.c:1842:osd_index_restore()) lustre-MDT0001: restore index '0x2010000' with [0x200000003:0x6:0x0]: rc = 0 [ 5396.706166] Lustre: 259241:0:(osd_scrub.c:1842:osd_index_restore()) lustre-MDT0001: restore index '0x1010000' with [0x200000003:0x5:0x0]: rc = 0 [ 5396.710486] Lustre: 259241:0:(osd_scrub.c:1842:osd_index_restore()) lustre-MDT0001: restore index '0x1010000-MDT0001' with [0x200000005:0x2:0x0]: rc = 0 [ 5396.714613] Lustre: 259241:0:(osd_scrub.c:1842:osd_index_restore()) lustre-MDT0001: restore index '0x2010000-MDT0001' with [0x200000005:0x3:0x0]: rc = 0 [ 5396.718343] Lustre: 259241:0:(osd_scrub.c:1842:osd_index_restore()) lustre-MDT0001: restore index '0x10000-MDT0001' with [0x200000005:0x1:0x0]: rc = 0 [ 5396.725495] Lustre: Found index 1 for lustre-MDT0001, updating log [ 5396.798668] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 5396.823387] Lustre: lustre-MDT0001: Aborting client recovery [ 5397.150508] loop2: detected capacity change from 0 to 204800000 [ 5397.200406] LDISKFS-fs (loop2): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 5397.240110] LDISKFS-fs (loop2): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 5397.266759] Lustre: 259325:0:(osd_scrub.c:1842:osd_index_restore()) lustre-OST0000: restore index 'fld' with [0x200000001:0x3:0x0]: rc = 0 [ 5397.270465] Lustre: 259325:0:(osd_scrub.c:1842:osd_index_restore()) lustre-OST0000: restore index '0x1020000' with [0x200000003:0x6:0x0]: rc = 0 [ 5397.273608] Lustre: 259325:0:(osd_scrub.c:1842:osd_index_restore()) lustre-OST0000: restore index '0x2020000' with [0x200000003:0x7:0x0]: rc = 0 [ 5397.277949] Lustre: 259325:0:(osd_scrub.c:1842:osd_index_restore()) lustre-OST0000: restore index '0x20000' with [0x200000003:0x5:0x0]: rc = 0 [ 5397.280963] Lustre: 259325:0:(osd_scrub.c:1842:osd_index_restore()) lustre-OST0000: restore index '0x20000-OST0000' with [0x200000005:0x1:0x0]: rc = 0 [ 5397.284324] Lustre: 259325:0:(osd_scrub.c:1842:osd_index_restore()) lustre-OST0000: restore index '0x2020000-OST0000' with [0x200000005:0x3:0x0]: rc = 0 [ 5397.287829] Lustre: 259325:0:(osd_scrub.c:1842:osd_index_restore()) lustre-OST0000: restore index '0x1020000-OST0000' with [0x200000005:0x2:0x0]: rc = 0 [ 5397.334311] Lustre: Found index 0 for lustre-OST0000, updating log [ 5397.369957] Lustre: lustre-OST0000: Aborting recovery [ 5397.678625] loop3: detected capacity change from 0 to 204800000 [ 5397.708085] LDISKFS-fs (loop3): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 5397.738896] LDISKFS-fs (loop3): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 5397.758319] Lustre: 259413:0:(osd_scrub.c:1842:osd_index_restore()) lustre-OST0001: restore index 'fld' with [0x200000001:0x3:0x0]: rc = 0 [ 5397.762770] Lustre: 259413:0:(osd_scrub.c:1842:osd_index_restore()) lustre-OST0001: restore index '0x2020000-OST0001' with [0x200000005:0x3:0x0]: rc = 0 [ 5397.765648] Lustre: 259413:0:(osd_scrub.c:1842:osd_index_restore()) lustre-OST0001: restore index '0x20000' with [0x200000003:0x5:0x0]: rc = 0 [ 5397.768646] Lustre: 259413:0:(osd_scrub.c:1842:osd_index_restore()) lustre-OST0001: restore index '0x1020000' with [0x200000003:0x6:0x0]: rc = 0 [ 5397.771678] Lustre: 259413:0:(osd_scrub.c:1842:osd_index_restore()) lustre-OST0001: restore index '0x1020000-OST0001' with [0x200000005:0x2:0x0]: rc = 0 [ 5397.774882] Lustre: 259413:0:(osd_scrub.c:1842:osd_index_restore()) lustre-OST0001: restore index '0x20000-OST0001' with [0x200000005:0x1:0x0]: rc = 0 [ 5397.778284] Lustre: 259413:0:(osd_scrub.c:1842:osd_index_restore()) lustre-OST0001: restore index '0x2020000' with [0x200000003:0x7:0x0]: rc = 0 [ 5397.810167] Lustre: lustre-OST0001: Imperative Recovery not enabled, recovery window 60-180 [ 5397.811910] Lustre: Skipped 1 previous similar message [ 5403.113334] LustreError: 259184:0:(lod_dev.c:510:lod_sub_recovery_thread()) lustre-MDT0001-osp-MDT0000: get update log duration 7, retries 0, failed: rc = -108 [ 5406.180049] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x0:6 to 0x0:33) [ 5406.186096] Lustre: lustre-OST0001: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x0:6 to 0x0:33) [ 5406.188407] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000bd0 [ 5406.195525] Lustre: lustre-OST0001-osc-MDT0000: update sequence from 0x100010000 to 0x2c0000bd0 [ 5407.203255] Lustre: lustre-OST0000: new connection from lustre-MDT0001-mdtlov (cleaning up unused objects from 0x280000400:5 to 0x280000400:33) [ 5407.203660] Lustre: lustre-OST0001: new connection from lustre-MDT0001-mdtlov (cleaning up unused objects from 0x2c0000400:5 to 0x2c0000400:33) [ 5408.294104] LustreError: 259151:0:(mdt_identity.c:115:mdt_identity_do_upcall()) lustre-MDT0000: error invoking upcall /root/Work/Lustre/R5/lustre-release/lustre/tests/../utils/l_getidentity lustre-MDT0000 0: rc -2; check /proc/fs/lustre/mdt/lustre-MDT0000/identity_upcall, time 247us: rc = -2 [ 5422.561676] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 5422.561874] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5422.562329] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 5422.570075] Lustre: Skipped 3 previous similar messages [ 5425.599278] LustreError: 259770:0:(obd_class.h:479:obd_check_dev()) Device 10 not setup [ 5425.601274] LustreError: 259770:0:(obd_class.h:479:obd_check_dev()) Skipped 1 previous similar message [ 5425.646971] Lustre: server umount lustre-MDT0000 complete [ 5425.970359] LustreError: 259720:0:(ldlm_lockd.c:2607:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1758742412 with bad export cookie 726337841503071604 [ 5425.972293] LustreError: MGC192.168.201.158@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 5425.973822] LustreError: 259720:0:(ldlm_lockd.c:2607:ldlm_cancel_handler()) Skipped 2 previous similar messages [ 5427.680683] LustreError: 259149:0:(ldlm_lib.c:1132:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 5427.681074] Lustre: lustre-MDT0001-lwp-OST0001: Connection to lustre-MDT0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5427.681375] Lustre: lustre-MDT0001: Not available for connect from 0@lo (stopping) [ 5427.681380] Lustre: Skipped 3 previous similar messages [ 5427.686327] LustreError: 259149:0:(ldlm_lib.c:1132:target_handle_connect()) Skipped 3 previous similar messages [ 5427.690475] Lustre: Skipped 1 previous similar message [ 5430.688525] LustreError: 259149:0:(ldlm_lib.c:1132:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 5430.688668] Lustre: lustre-MDT0001: Not available for connect from 0@lo (stopping) [ 5430.693743] LustreError: 259149:0:(ldlm_lib.c:1132:target_handle_connect()) Skipped 1 previous similar message [ 5430.695486] Lustre: Skipped 2 previous similar messages [ 5432.121838] LustreError: 259823:0:(obd_class.h:479:obd_check_dev()) Device 18 not setup [ 5432.123945] LustreError: 259823:0:(obd_class.h:479:obd_check_dev()) Skipped 7 previous similar messages [ 5432.211700] Lustre: server umount lustre-MDT0001 complete [ 5438.634345] LustreError: 259875:0:(obd_class.h:479:obd_check_dev()) Device 23 not setup [ 5438.637718] LustreError: 259875:0:(obd_class.h:479:obd_check_dev()) Skipped 7 previous similar messages [ 5438.667365] Lustre: server umount lustre-OST0000 complete [ 5453.280144] Lustre: lustre-OST0001 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 5453.298842] LustreError: 259926:0:(obd_class.h:479:obd_check_dev()) Device 2 not setup [ 5453.301050] LustreError: 259926:0:(obd_class.h:479:obd_check_dev()) Skipped 3 previous similar messages [ 5453.401841] Lustre: server umount lustre-OST0001 complete [ 5456.297767] Lustre: DEBUG MARKER: == conf-sanity test 109a: test lctl clear_conf fsname ==== 15:34:03 (1758742443) [ 5465.549436] Lustre: DEBUG MARKER: oleg158-server.virtnet: executing set_hostid [ 5468.918987] Lustre: DEBUG MARKER: oleg158-server.virtnet: executing load_modules_local [ 5472.634218] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 5474.970186] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 5476.989811] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 5479.112826] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 5481.454859] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 5481.485106] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 5481.581044] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 5481.595652] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 5481.632613] Lustre: lustre-MDT0000: new disk, initializing [ 5481.657376] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 5481.664408] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 5482.980198] Lustre: DEBUG MARKER: oleg158-server.virtnet: executing set_default_debug -1 all [ 5487.127723] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 5487.153331] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 5487.183186] Lustre: 263472:0:(mgs_llog.c:1345:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 5487.209214] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [ 5487.212392] Lustre: Skipped 1 previous similar message [ 5487.248518] Lustre: lustre-MDT0001: new disk, initializing [ 5487.286052] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 5487.300102] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [ 5487.303094] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 5488.627241] Lustre: DEBUG MARKER: oleg158-server.virtnet: executing set_default_debug -1 all [ 5491.739881] Lustre: DEBUG MARKER: oleg158-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 5493.023353] Lustre: DEBUG MARKER: oleg158-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 5495.334524] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 5495.359242] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 5495.455600] Lustre: lustre-OST0000: new disk, initializing [ 5495.458095] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 5495.488206] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 5496.950540] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [ 5496.954116] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 5496.992371] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 5497.384635] Lustre: DEBUG MARKER: oleg158-server.virtnet: executing set_default_debug -1 all [ 5500.456711] Lustre: DEBUG MARKER: oleg158-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 5502.814858] Lustre: Setting parameter lustre-MDT0000.mdd.atime_diff=62 in log lustre-MDT0000 [ 5507.868884] Lustre: Modifying parameter lustre-MDT0000.mdd.atime_diff=63 in log lustre-MDT0000 [ 5516.245846] Lustre: Setting parameter lustre-client.llite.max_read_ahead_mb=32 in log lustre-client [ 5558.756592] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5558.757280] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 5558.760954] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 5558.763269] LustreError: Skipped 1 previous similar message [ 5558.767133] Lustre: Skipped 1 previous similar message [ 5562.220810] LustreError: 267151:0:(obd_class.h:479:obd_check_dev()) Device 19 not setup [ 5562.222764] LustreError: 267151:0:(obd_class.h:479:obd_check_dev()) Skipped 5 previous similar messages [ 5562.291199] Lustre: server umount lustre-OST0000 complete [ 5563.873403] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5563.878947] Lustre: Skipped 1 previous similar message [ 5563.881409] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 5563.883112] Lustre: Skipped 1 previous similar message [ 5569.626310] Lustre: server umount lustre-MDT0000 complete [ 5570.803396] LustreError: 264252:0:(ldlm_lockd.c:2607:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1758742557 with bad export cookie 726337841503074313 [ 5570.808224] LustreError: MGC192.168.201.158@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 5570.812444] LustreError: 264252:0:(ldlm_lockd.c:2607:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 5570.953170] Lustre: server umount lustre-MDT0001 complete [ 5573.753131] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 5575.013975] Lustre: DEBUG MARKER: oleg158-server.virtnet: executing set_default_debug -1 all [ 5576.621155] Lustre: server umount MGS complete [ 5580.257442] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 5580.398607] LustreError: 269117:0:(ldlm_lib.c:1132:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 5580.406540] LustreError: 269129:0:(qmt_pool.c:1513:qmt_pool_add_rem()) lustre-QMT0000: can't remove lustre-OST0000_UUID pool 'pool1': rc = -22 [ 5580.432344] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 5581.628421] Lustre: DEBUG MARKER: oleg158-server.virtnet: executing set_default_debug -1 all [ 5584.449549] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 5585.928582] Lustre: DEBUG MARKER: oleg158-server.virtnet: executing set_default_debug -1 all [ 5587.763522] Lustre: DEBUG MARKER: oleg158-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 5589.156859] Lustre: DEBUG MARKER: oleg158-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 5591.636241] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 5593.874434] Lustre: DEBUG MARKER: oleg158-server.virtnet: executing set_default_debug -1 all [ 5595.821511] Lustre: DEBUG MARKER: oleg158-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 5626.850407] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 5626.851099] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5626.851847] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 5626.851852] Lustre: Skipped 2 previous similar messages [ 5626.862734] Lustre: Skipped 2 previous similar messages [ 5631.085302] LustreError: 271796:0:(obd_class.h:479:obd_check_dev()) Device 21 not setup [ 5631.086951] LustreError: 271796:0:(obd_class.h:479:obd_check_dev()) Skipped 19 previous similar messages [ 5631.143293] Lustre: server umount lustre-OST0000 complete [ 5636.065078] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 5636.067394] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5638.564218] Lustre: server umount lustre-MDT0000 complete [ 5639.812028] LustreError: 269099:0:(ldlm_lockd.c:2607:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1758742626 with bad export cookie 726337841503076126 [ 5639.815085] LustreError: MGC192.168.201.158@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 5639.816181] LustreError: 269099:0:(ldlm_lockd.c:2607:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 5639.982225] Lustre: server umount lustre-MDT0001 complete [ 5643.909695] Lustre: DEBUG MARKER: oleg158-server.virtnet: executing unload_modules_local [ 5644.989389] Key type lgssc unregistered [ 5645.136376] LNet: 272729:0:(lib-ptl.c:966:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 5645.138696] LNetError: 272729:0:(acceptor.c:263:lnet_acceptor_remove_socket()) Interface ens2 not found [ 5645.152306] LNet: Removed LNI 192.168.201.158@tcp [ 5645.417592] Key type .llcrypt unregistered [ 5645.418720] Key type ._llcrypt unregistered [ 5657.735631] Key type ._llcrypt registered [ 5657.738316] Key type .llcrypt registered [ 5657.777117] Lustre: DEBUG MARKER: oleg158-server.virtnet: executing set_hostid [ 5663.893749] Lustre: DEBUG MARKER: oleg158-server.virtnet: executing load_modules_local [ 5664.218647] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 5664.260068] alg: No test for adler32 (adler32-zlib) [ 5665.119639] Lustre: Lustre: Build Version: 2.16.58_53_g2e3f3ed [ 5665.193373] LNet: Added LNI 192.168.201.158@tcp [8/256/0/180] [ 5666.776182] Key type lgssc registered [ 5667.150200] Lustre: Echo OBD driver; http://www.lustre.org/ [ 5670.160651] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 5672.354621] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 5674.242908] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 5676.376440] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 5678.832764] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 5678.850762] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 5678.856514] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 5679.951116] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 5679.966142] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 5680.006707] Lustre: lustre-MDT0000: new disk, initializing [ 5680.032657] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 5680.039331] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 5681.417671] Lustre: DEBUG MARKER: oleg158-server.virtnet: executing set_default_debug -1 all [ 5685.664961] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 5685.689045] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 5685.719951] Lustre: 276632:0:(mgs_llog.c:1345:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 5685.746464] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [ 5685.748486] Lustre: Skipped 1 previous similar message [ 5685.787383] Lustre: lustre-MDT0001: new disk, initializing [ 5685.830686] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 5685.846532] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [ 5685.851454] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 5687.179230] Lustre: DEBUG MARKER: oleg158-server.virtnet: executing set_default_debug -1 all [ 5690.335963] Lustre: DEBUG MARKER: oleg158-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 5691.602565] Lustre: DEBUG MARKER: oleg158-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 5693.947236] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 5693.969330] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 5694.083723] Lustre: lustre-OST0000: new disk, initializing [ 5694.085528] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 5694.122064] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 5695.965195] Lustre: DEBUG MARKER: oleg158-server.virtnet: executing set_default_debug -1 all [ 5696.954800] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [ 5696.957731] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 5696.986180] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 5699.039443] Lustre: DEBUG MARKER: oleg158-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 5702.398527] Lustre: Setting parameter lustre-MDT0000.mdd.atime_diff=62 in log lustre-MDT0000 [ 5706.223660] Lustre: Modifying parameter lustre-MDT0000.mdd.atime_diff=63 in log lustre-MDT0000 [ 5713.333868] Lustre: Setting parameter lustre-client.llite.max_read_ahead_mb=32 in log lustre-client [ 5719.862236] Lustre: Modifying parameter lustre-client.llite.max_read_ahead_mb=64 in log lustre-client [ 5758.434862] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 5758.435347] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5758.436037] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 5758.441756] Lustre: Skipped 1 previous similar message [ 5763.556504] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 5763.558936] Lustre: Skipped 2 previous similar messages [ 5768.676824] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 5768.679221] Lustre: Skipped 1 previous similar message [ 5772.256213] Lustre: lustre-OST0000 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 5772.276108] LustreError: 280197:0:(obd_class.h:479:obd_check_dev()) Device 19 not setup [ 5772.348279] Lustre: server umount lustre-OST0000 complete [ 5773.793856] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5773.794808] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 5773.799180] Lustre: Skipped 1 previous similar message [ 5773.804085] Lustre: Skipped 1 previous similar message [ 5778.913132] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 5778.915605] Lustre: Skipped 1 previous similar message [ 5788.128129] Lustre: lustre-MDT0000 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 5788.163892] LustreError: 280399:0:(obd_class.h:479:obd_check_dev()) Device 22 not setup [ 5788.165582] LustreError: 280399:0:(obd_class.h:479:obd_check_dev()) Skipped 3 previous similar messages [ 5788.190466] Lustre: server umount lustre-MDT0000 complete [ 5789.469371] LustreError: 276623:0:(ldlm_lockd.c:2607:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1758742775 with bad export cookie 11824224447099759366 [ 5789.472256] LustreError: MGC192.168.201.158@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 5789.473529] LustreError: 276623:0:(ldlm_lockd.c:2607:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 5789.535108] LustreError: 280604:0:(obd_class.h:479:obd_check_dev()) Device 21 not setup [ 5789.536660] LustreError: 280604:0:(obd_class.h:479:obd_check_dev()) Skipped 5 previous similar messages [ 5789.625623] Lustre: server umount lustre-MDT0001 complete [ 5792.567084] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 5793.889617] Lustre: DEBUG MARKER: oleg158-server.virtnet: executing set_default_debug -1 all [ 5795.434378] LustreError: 281544:0:(obd_class.h:479:obd_check_dev()) Device 2 not setup [ 5795.436262] LustreError: 281544:0:(obd_class.h:479:obd_check_dev()) Skipped 7 previous similar messages [ 5795.457408] Lustre: server umount MGS complete [ 5799.084188] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 5799.241409] LustreError: 282167:0:(ldlm_lib.c:1132:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 5799.248745] LustreError: 282179:0:(qmt_pool.c:1513:qmt_pool_add_rem()) lustre-QMT0000: can't remove lustre-OST0000_UUID pool 'pool1': rc = -22 [ 5799.252205] LustreError: 282167:0:(ldlm_lib.c:1132:target_handle_connect()) Skipped 3 previous similar messages [ 5799.268719] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 5800.601902] Lustre: DEBUG MARKER: oleg158-server.virtnet: executing set_default_debug -1 all [ 5803.501651] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 5803.628333] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 5804.898602] Lustre: DEBUG MARKER: oleg158-server.virtnet: executing set_default_debug -1 all [ 5806.659859] Lustre: DEBUG MARKER: oleg158-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 5807.825278] Lustre: DEBUG MARKER: oleg158-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 5810.025747] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 5811.894170] Lustre: DEBUG MARKER: oleg158-server.virtnet: executing set_default_debug -1 all [ 5813.671668] Lustre: DEBUG MARKER: oleg158-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 5833.696804] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 5833.699224] LustreError: Skipped 1 previous similar message [ 5833.700450] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5833.704562] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 5833.707661] Lustre: Skipped 2 previous similar messages [ 5834.721909] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 5837.357119] LustreError: 284823:0:(obd_class.h:479:obd_check_dev()) Device 21 not setup [ 5837.359205] LustreError: 284823:0:(obd_class.h:479:obd_check_dev()) Skipped 1 previous similar message [ 5837.424587] Lustre: server umount lustre-OST0000 complete [ 5839.328964] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 5839.330866] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5839.335182] Lustre: Skipped 1 previous similar message [ 5844.823632] Lustre: server umount lustre-MDT0000 complete [ 5844.961099] LustreError: 282167:0:(ldlm_lib.c:1132:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 5844.967185] LustreError: 282167:0:(ldlm_lib.c:1132:target_handle_connect()) Skipped 1 previous similar message [ 5846.038090] LustreError: 282148:0:(ldlm_lockd.c:2607:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1758742832 with bad export cookie 11824224447099761291 [ 5846.040607] LustreError: MGC192.168.201.158@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 5846.042576] LustreError: 282148:0:(ldlm_lockd.c:2607:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 5846.102930] LustreError: 285225:0:(obd_class.h:479:obd_check_dev()) Device 16 not setup [ 5846.104493] LustreError: 285225:0:(obd_class.h:479:obd_check_dev()) Skipped 9 previous similar messages [ 5846.190854] Lustre: server umount lustre-MDT0001 complete [ 5849.689329] Lustre: DEBUG MARKER: oleg158-server.virtnet: executing unload_modules_local [ 5850.750801] Key type lgssc unregistered [ 5850.880293] LNet: 285756:0:(lib-ptl.c:966:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 5850.882778] LNetError: 285756:0:(acceptor.c:263:lnet_acceptor_remove_socket()) Interface ens2 not found [ 5850.888327] LNet: Removed LNI 192.168.201.158@tcp [ 5851.190661] Key type .llcrypt unregistered [ 5851.191839] Key type ._llcrypt unregistered [ 5857.624878] Key type ._llcrypt registered [ 5857.625983] Key type .llcrypt registered [ 5857.664312] Lustre: DEBUG MARKER: oleg158-server.virtnet: executing set_hostid [ 5863.286259] Lustre: DEBUG MARKER: oleg158-server.virtnet: executing load_modules_local [ 5863.634524] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 5863.664799] alg: No test for adler32 (adler32-zlib) [ 5864.517469] Lustre: Lustre: Build Version: 2.16.58_53_g2e3f3ed [ 5864.611555] LNet: Added LNI 192.168.201.158@tcp [8/256/0/180] [ 5866.200157] Key type lgssc registered [ 5866.553739] Lustre: Echo OBD driver; http://www.lustre.org/ [ 5869.417299] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 5871.312291] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 5874.098675] Lustre: DEBUG MARKER: oleg158-server.virtnet: executing load_modules_local [ 5876.939845] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 5879.131087] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 5879.154875] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 5879.161328] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 5880.260503] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 5880.274704] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 5880.314443] Lustre: lustre-MDT0000: new disk, initializing [ 5880.341840] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 5880.349401] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 5881.715979] Lustre: DEBUG MARKER: oleg158-server.virtnet: executing set_default_debug -1 all [ 5885.810259] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 5885.835802] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 5885.936412] Lustre: lustre-OST0000: new disk, initializing [ 5885.938422] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 5885.941128] Lustre: Skipped 1 previous similar message [ 5885.963669] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 5887.956626] Lustre: DEBUG MARKER: oleg158-server.virtnet: executing set_default_debug -1 all [ 5893.120023] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:0:ost [ 5893.122710] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:0:ost] [ 5893.131855] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x240000400 [ 6077.409771] Lustre: lustre-MDT0000-lwp-OST0000: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 6077.414835] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 6082.047555] LustreError: 290670:0:(obd_class.h:479:obd_check_dev()) Device 13 not setup [ 6082.194391] Lustre: server umount lustre-MDT0000 complete [ 6083.367416] LustreError: 289719:0:(ldlm_lockd.c:2607:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1758743069 with bad export cookie 14366265653486041743 [ 6083.369075] LustreError: MGC192.168.201.158@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 6083.370740] LustreError: 289719:0:(ldlm_lockd.c:2607:ldlm_cancel_handler()) Skipped 2 previous similar messages [ 6083.391990] LustreError: 290871:0:(obd_class.h:479:obd_check_dev()) Device 2 not setup [ 6083.394315] LustreError: 290871:0:(obd_class.h:479:obd_check_dev()) Skipped 3 previous similar messages [ 6083.460569] Lustre: server umount lustre-OST0000 complete [ 6091.252186] Lustre: DEBUG MARKER: oleg158-server.virtnet: executing set_hostid [ 6094.372652] Lustre: DEBUG MARKER: oleg158-server.virtnet: executing load_modules_local [ 6097.754174] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 6100.051754] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 6102.100546] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 6104.276582] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 6106.544316] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 6106.566134] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 6106.646695] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 6106.659817] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 6106.694698] Lustre: lustre-MDT0000: new disk, initializing [ 6106.718777] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 6106.724950] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 6107.997574] Lustre: DEBUG MARKER: oleg158-server.virtnet: executing set_default_debug -1 all [ 6112.204516] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 6112.227725] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 6112.260319] Lustre: 293908:0:(mgs_llog.c:1345:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 6112.281500] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [ 6112.282894] Lustre: Skipped 1 previous similar message [ 6112.317494] Lustre: lustre-MDT0001: new disk, initializing [ 6112.350756] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 6112.361241] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [ 6112.364839] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 6113.678224] Lustre: DEBUG MARKER: oleg158-server.virtnet: executing set_default_debug -1 all [ 6116.838540] Lustre: DEBUG MARKER: oleg158-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 6118.072143] Lustre: DEBUG MARKER: oleg158-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 6120.346294] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 6120.368320] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 6120.454514] Lustre: lustre-OST0000: new disk, initializing [ 6120.456121] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 6120.482951] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 6120.847522] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [ 6120.851930] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 6120.893576] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 6122.387316] Lustre: DEBUG MARKER: oleg158-server.virtnet: executing set_default_debug -1 all [ 6125.455280] Lustre: DEBUG MARKER: oleg158-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 6126.902403] Lustre: DEBUG MARKER: oleg158-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 [ 6126.978708] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [ 6128.338353] Lustre: DEBUG MARKER: oleg158-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid 50 [ 6128.418424] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [ 6131.119067] LustreError: 296847:0:(obd_class.h:479:obd_check_dev()) Device 19 not setup [ 6131.122057] LustreError: 296847:0:(obd_class.h:479:obd_check_dev()) Skipped 3 previous similar messages [ 6131.205682] Lustre: server umount lustre-OST0000 complete [ 6132.704810] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 6132.707445] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 6132.711354] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 6136.289425] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 6136.289910] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 6141.409400] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 6141.411840] Lustre: Skipped 2 previous similar messages [ 6142.435578] LustreError: 297047:0:(obd_class.h:479:obd_check_dev()) Device 22 not setup [ 6142.437655] LustreError: 297047:0:(obd_class.h:479:obd_check_dev()) Skipped 3 previous similar messages [ 6142.460522] Lustre: server umount lustre-MDT0000 complete [ 6143.660166] LustreError: 294688:0:(ldlm_lockd.c:2607:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1758743129 with bad export cookie 14366265653488982534 [ 6143.661608] LustreError: MGC192.168.201.158@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 6143.663800] LustreError: 294688:0:(ldlm_lockd.c:2607:ldlm_cancel_handler()) Skipped 1 previous similar message [ 6153.187953] LustreError: 297250:0:(obd_class.h:479:obd_check_dev()) Device 21 not setup [ 6153.189697] LustreError: 297250:0:(obd_class.h:479:obd_check_dev()) Skipped 5 previous similar messages [ 6153.257039] Lustre: server umount lustre-MDT0001 complete [ 6154.354961] Lustre: DEBUG MARKER: SKIP: conf-sanity test_111 skipping SLOW test 111 [ 6154.902365] Lustre: DEBUG MARKER: == conf-sanity test 112a: mount OST with no_create option ========================================================== 15:45:41 (1758743141) [ 6157.048496] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 6157.229868] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 6158.455919] Lustre: DEBUG MARKER: oleg158-server.virtnet: executing set_default_debug -1 all [ 6161.135786] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 6162.529813] Lustre: DEBUG MARKER: oleg158-server.virtnet: executing set_default_debug -1 all [ 6164.216104] Lustre: DEBUG MARKER: oleg158-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 6165.421232] Lustre: DEBUG MARKER: oleg158-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 6167.671027] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 6169.638702] Lustre: DEBUG MARKER: oleg158-server.virtnet: executing set_default_debug -1 all [ 6171.318740] Lustre: DEBUG MARKER: oleg158-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 6173.477405] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 6173.500208] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 6173.544912] Lustre: lustre-OST0001: new disk, initializing [ 6173.547623] Lustre: srv-lustre-OST0001: No data found on store. Initialize space. [ 6173.575839] Lustre: lustre-OST0001: Imperative Recovery not enabled, recovery window 60-180 [ 6173.578691] Lustre: Skipped 2 previous similar messages [ 6174.336591] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x00000002c0000400-0x0000000300000400]:1:ost [ 6174.341475] Lustre: cli-lustre-OST0001-super: Allocated super-sequence [0x00000002c0000400-0x0000000300000400]:1:ost] [ 6174.357452] LustreError: 300465:0:(osp_precreate.c:969:osp_precreate_cleanup_orphans()) lustre-OST0001-osc-MDT0001: cannot cleanup orphans: rc = -1 [ 6175.396554] LustreError: 300473:0:(osp_precreate.c:969:osp_precreate_cleanup_orphans()) lustre-OST0001-osc-MDT0000: cannot cleanup orphans: rc = -1 [ 6175.397741] Lustre: DEBUG MARKER: oleg158-server.virtnet: executing set_default_debug -1 all [ 6175.400196] LustreError: 300473:0:(osp_precreate.c:969:osp_precreate_cleanup_orphans()) Skipped 2 previous similar messages [ 6176.419173] LustreError: 300473:0:(osp_precreate.c:969:osp_precreate_cleanup_orphans()) lustre-OST0001-osc-MDT0000: cannot cleanup orphans: rc = -1 [ 6176.423238] LustreError: 300473:0:(osp_precreate.c:969:osp_precreate_cleanup_orphans()) Skipped 1 previous similar message [ 6178.467038] LustreError: 300465:0:(osp_precreate.c:969:osp_precreate_cleanup_orphans()) lustre-OST0001-osc-MDT0001: cannot cleanup orphans: rc = -1 [ 6178.469761] LustreError: 300465:0:(osp_precreate.c:969:osp_precreate_cleanup_orphans()) Skipped 3 previous similar messages [ 6178.937105] Lustre: DEBUG MARKER: oleg158-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 [ 6182.574638] LustreError: 300465:0:(osp_precreate.c:969:osp_precreate_cleanup_orphans()) lustre-OST0001-osc-MDT0001: cannot cleanup orphans: rc = -1 [ 6182.578278] LustreError: 300465:0:(osp_precreate.c:969:osp_precreate_cleanup_orphans()) Skipped 7 previous similar messages [ 6183.098690] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 4 sec [ 6184.182448] Lustre: DEBUG MARKER: oleg158-client.virtnet: executing wait_import_state (FULL|IDLE) osc.lustre-OST0000-osc-ffff946ec9709000.ost_server_uuid 50 [ 6184.677934] Lustre: DEBUG MARKER: osc.lustre-OST0000-osc-ffff946ec9709000.ost_server_uuid in FULL state after 0 sec [ 6185.823830] Lustre: DEBUG MARKER: oleg158-client.virtnet: executing wait_import_state (FULL|IDLE) osc.lustre-OST0001-osc-ffff946ec9709000.ost_server_uuid 50 [ 6186.342623] Lustre: DEBUG MARKER: osc.lustre-OST0001-osc-ffff946ec9709000.ost_server_uuid in FULL state after 0 sec [ 6187.688055] Lustre: lustre-OST0001-osc-MDT0000: update sequence from 0x100010000 to 0x2c0000401 [ 6198.241608] LustreError: lustre-OST0001-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 6198.241702] Lustre: lustre-OST0001-osc-MDT0000: Connection to lustre-OST0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 6198.244754] LustreError: Skipped 1 previous similar message [ 6198.248396] Lustre: lustre-OST0001: Not available for connect from 0@lo (stopping) [ 6198.251593] Lustre: Skipped 1 previous similar message [ 6203.951078] LustreError: 301650:0:(obd_class.h:479:obd_check_dev()) Device 25 not setup [ 6203.953438] LustreError: 301650:0:(obd_class.h:479:obd_check_dev()) Skipped 7 previous similar messages [ 6203.971041] Lustre: server umount lustre-OST0001 complete [ 6206.434608] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 6206.436330] LustreError: 301550:0:(ldlm_lib.c:1132:target_handle_connect()) lustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 6206.437342] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 6206.437347] Lustre: Skipped 4 previous similar messages [ 6206.438418] Lustre: Skipped 2 previous similar messages [ 6206.442420] LustreError: 301550:0:(ldlm_lib.c:1132:target_handle_connect()) Skipped 3 previous similar messages [ 6211.559794] LustreError: 301550:0:(ldlm_lib.c:1132:target_handle_connect()) lustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 6211.564773] LustreError: 301550:0:(ldlm_lib.c:1132:target_handle_connect()) Skipped 1 previous similar message [ 6211.695563] Lustre: server umount lustre-OST0000 complete [ 6216.673921] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 6216.677559] Lustre: Skipped 1 previous similar message [ 6219.096302] Lustre: server umount lustre-MDT0000 complete [ 6220.386337] LustreError: 297804:0:(ldlm_lockd.c:2607:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1758743206 with bad export cookie 14366265653488983402 [ 6220.391692] LustreError: MGC192.168.201.158@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 6220.457817] LustreError: 302256:0:(obd_class.h:479:obd_check_dev()) Device 27 not setup [ 6220.460972] LustreError: 302256:0:(obd_class.h:479:obd_check_dev()) Skipped 15 previous similar messages [ 6220.558744] Lustre: server umount lustre-MDT0001 complete [ 6224.369333] Lustre: DEBUG MARKER: oleg158-server.virtnet: executing unload_modules_local [ 6225.570585] Key type lgssc unregistered [ 6225.720394] LNet: 302787:0:(lib-ptl.c:966:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 6225.723442] LNetError: 302787:0:(acceptor.c:263:lnet_acceptor_remove_socket()) Interface ens2 not found [ 6225.735349] LNet: Removed LNI 192.168.201.158@tcp [ 6226.080469] Key type .llcrypt unregistered [ 6226.081496] Key type ._llcrypt unregistered [ 6235.485889] Key type ._llcrypt registered [ 6235.486884] Key type .llcrypt registered [ 6235.529123] Lustre: DEBUG MARKER: oleg158-server.virtnet: executing load_modules_local [ 6235.934186] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 6235.994206] alg: No test for adler32 (adler32-zlib) [ 6236.872119] Lustre: Lustre: Build Version: 2.16.58_53_g2e3f3ed [ 6236.967415] LNet: Added LNI 192.168.201.158@tcp [8/256/0/180] [ 6238.552122] Key type lgssc registered [ 6238.983399] Lustre: Echo OBD driver; http://www.lustre.org/ [ 6241.743313] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 6241.750658] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 6242.919126] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 6244.318251] Lustre: DEBUG MARKER: oleg158-server.virtnet: executing set_default_debug -1 all [ 6246.179420] Lustre: DEBUG MARKER: oleg158-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 6248.518817] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 6248.658945] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 6250.021920] Lustre: DEBUG MARKER: oleg158-server.virtnet: executing set_default_debug -1 all [ 6253.239175] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 6253.354819] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 6255.332507] Lustre: DEBUG MARKER: oleg158-server.virtnet: executing set_default_debug -1 all [ 6257.245497] Lustre: DEBUG MARKER: oleg158-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 6258.472396] LustreError: 305716:0:(ldlm_lib.c:1132:target_handle_connect()) lustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 6258.480351] LustreError: 305716:0:(ldlm_lib.c:1132:target_handle_connect()) Skipped 2 previous similar messages [ 6259.634069] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 6259.717414] Lustre: lustre-OST0001: Imperative Recovery not enabled, recovery window 60-180 [ 6261.768481] Lustre: DEBUG MARKER: oleg158-server.virtnet: executing set_default_debug -1 all [ 6263.727827] Lustre: DEBUG MARKER: oleg158-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0001-osc-[-0-9a-f]*.ost_server_uuid [ 6268.073993] Lustre: lustre-OST0001: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x2c0000401:3 to 0x2c0000401:33) [ 6268.074075] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:36 to 0x280000401:65) [ 6270.338428] Lustre: DEBUG MARKER: oleg158-server.virtnet: executing wait_import_state (FULL|IDLE) os[cp].lustre-OST0001-osc-MDT0001.ost_server_uuid 50 [ 6270.407792] Lustre: DEBUG MARKER: os[cp].lustre-OST0001-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [ 6347.232819] LustreError: lustre-OST0001-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 6347.235707] Lustre: lustre-OST0001-osc-MDT0001: Connection to lustre-OST0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 6347.240982] Lustre: lustre-OST0001: Not available for connect from 0@lo (stopping) [ 6350.306872] Lustre: lustre-OST0001: Not available for connect from 0@lo (stopping) [ 6350.309199] Lustre: lustre-OST0001-osc-MDT0000: Connection to lustre-OST0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 6350.312476] LustreError: lustre-OST0001-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 6352.364093] LustreError: 307585:0:(obd_class.h:479:obd_check_dev()) Device 27 not setup [ 6352.380057] Lustre: server umount lustre-OST0001 complete [ 6355.426927] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 6355.427602] LustreError: 306430:0:(ldlm_lib.c:1132:target_handle_connect()) lustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 6355.427663] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 6355.427671] Lustre: Skipped 1 previous similar message [ 6355.430850] Lustre: Skipped 1 previous similar message [ 6355.434996] LustreError: 306430:0:(ldlm_lib.c:1132:target_handle_connect()) Skipped 1 previous similar message [ 6355.436747] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 6360.547813] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 6360.547851] LustreError: 306430:0:(ldlm_lib.c:1132:target_handle_connect()) lustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 6360.549379] Lustre: Skipped 2 previous similar messages [ 6360.554859] LustreError: 306430:0:(ldlm_lib.c:1132:target_handle_connect()) Skipped 1 previous similar message [ 6365.666343] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 6365.666650] LustreError: 306430:0:(ldlm_lib.c:1132:target_handle_connect()) lustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 6365.672073] LustreError: 306430:0:(ldlm_lib.c:1132:target_handle_connect()) Skipped 1 previous similar message [ 6368.224107] Lustre: lustre-OST0000 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 6368.240053] LustreError: 307787:0:(obd_class.h:479:obd_check_dev()) Device 23 not setup [ 6368.241621] LustreError: 307787:0:(obd_class.h:479:obd_check_dev()) Skipped 3 previous similar messages [ 6368.297028] Lustre: server umount lustre-OST0000 complete [ 6370.786276] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 6370.789821] Lustre: Skipped 1 previous similar message [ 6375.553560] LustreError: 307989:0:(obd_class.h:479:obd_check_dev()) Device 10 not setup [ 6375.556270] LustreError: 307989:0:(obd_class.h:479:obd_check_dev()) Skipped 3 previous similar messages [ 6375.586346] Lustre: server umount lustre-MDT0000 complete [ 6375.906213] LustreError: 304165:0:(ldlm_lib.c:1132:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 6375.910283] LustreError: 304165:0:(ldlm_lib.c:1132:target_handle_connect()) Skipped 1 previous similar message [ 6376.849151] LustreError: 305738:0:(ldlm_lockd.c:2607:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1758743363 with bad export cookie 14038289414387566533 [ 6376.852131] LustreError: MGC192.168.201.158@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 6376.853031] LustreError: 305738:0:(ldlm_lockd.c:2607:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 6377.018334] Lustre: server umount lustre-MDT0001 complete [ 6381.177615] Lustre: DEBUG MARKER: oleg158-server.virtnet: executing unload_modules_local [ 6382.384813] Key type lgssc unregistered [ 6382.538673] LNet: 308721:0:(lib-ptl.c:966:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 6382.540555] LNetError: 308721:0:(acceptor.c:263:lnet_acceptor_remove_socket()) Interface ens2 not found [ 6382.550326] LNet: Removed LNI 192.168.201.158@tcp [ 6382.855799] Key type .llcrypt unregistered [ 6382.856885] Key type ._llcrypt unregistered [ 6399.898778] Key type ._llcrypt registered [ 6399.899841] Key type .llcrypt registered [ 6399.932866] Lustre: DEBUG MARKER: oleg158-server.virtnet: executing load_modules_local [ 6400.293772] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 6400.338167] alg: No test for adler32 (adler32-zlib) [ 6401.214865] Lustre: Lustre: Build Version: 2.16.58_53_g2e3f3ed [ 6401.306940] LNet: Added LNI 192.168.201.158@tcp [8/256/0/180] [ 6402.896231] Key type lgssc registered [ 6403.287253] Lustre: Echo OBD driver; http://www.lustre.org/ [ 6406.337234] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 6406.343534] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 6407.494711] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 6408.754810] Lustre: DEBUG MARKER: oleg158-server.virtnet: executing set_default_debug -1 all [ 6411.537769] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 6411.668733] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 6412.830844] Lustre: DEBUG MARKER: oleg158-server.virtnet: executing set_default_debug -1 all [ 6413.733588] Lustre: Modifying parameter general.debug_raw_pointers=Y in log params [ 6415.851391] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 6415.986179] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 6416.996446] LustreError: 312426:0:(ldlm_lib.c:1132:target_handle_connect()) lustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 6417.000399] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:36 to 0x280000401:97) [ 6417.003524] LustreError: 312426:0:(ldlm_lib.c:1132:target_handle_connect()) Skipped 2 previous similar messages [ 6417.991644] Lustre: DEBUG MARKER: oleg158-server.virtnet: executing set_default_debug -1 all [ 6421.057146] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 6421.098311] LustreError: 312426:0:(ldlm_lib.c:1132:target_handle_connect()) lustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 6421.103239] LustreError: 312426:0:(ldlm_lib.c:1132:target_handle_connect()) Skipped 1 previous similar message [ 6421.161585] Lustre: lustre-OST0001: Imperative Recovery not enabled, recovery window 60-180 [ 6423.133220] Lustre: DEBUG MARKER: oleg158-server.virtnet: executing set_default_debug -1 all [ 6423.869097] Lustre: lustre-OST0001: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x2c0000401:3 to 0x2c0000401:65) [ 6427.258077] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 6435.098195] Lustre: Setting parameter general.lod.*.mdt_hash=crush in log params [ 6447.075565] Lustre: lustre-MDT0000-lwp-OST0000: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 6447.077580] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 6447.079491] Lustre: Skipped 1 previous similar message [ 6447.082457] Lustre: Skipped 3 previous similar messages [ 6450.045757] LustreError: 314377:0:(obd_class.h:479:obd_check_dev()) Device 10 not setup [ 6450.075767] Lustre: server umount lustre-MDT0000 complete [ 6451.330610] LustreError: 313980:0:(ldlm_lockd.c:2607:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1758743437 with bad export cookie 17590846157992812251 [ 6451.334319] LustreError: 313980:0:(ldlm_lockd.c:2607:ldlm_cancel_handler()) Skipped 2 previous similar messages [ 6451.334746] LustreError: MGC192.168.201.158@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 6451.400350] LustreError: 314579:0:(obd_class.h:479:obd_check_dev()) Device 18 not setup [ 6451.402074] LustreError: 314579:0:(obd_class.h:479:obd_check_dev()) Skipped 7 previous similar messages [ 6451.483756] Lustre: server umount lustre-MDT0001 complete [ 6463.030530] LustreError: 314779:0:(obd_class.h:479:obd_check_dev()) Device 23 not setup [ 6463.033662] LustreError: 314779:0:(obd_class.h:479:obd_check_dev()) Skipped 7 previous similar messages [ 6463.051459] Lustre: server umount lustre-OST0000 complete [ 6474.610536] LustreError: 314981:0:(obd_class.h:479:obd_check_dev()) Device 2 not setup [ 6474.613509] LustreError: 314981:0:(obd_class.h:479:obd_check_dev()) Skipped 3 previous similar messages [ 6474.686217] Lustre: server umount lustre-OST0001 complete [ 6480.184514] Lustre: DEBUG MARKER: oleg158-server.virtnet: executing load_modules_local [ 6483.656090] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 6483.797086] LustreError: 316167:0:(ldlm_lib.c:1132:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 6483.818233] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 6485.107381] Lustre: DEBUG MARKER: oleg158-server.virtnet: executing set_default_debug -1 all [ 6488.021617] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 6489.481236] Lustre: DEBUG MARKER: oleg158-server.virtnet: executing set_default_debug -1 all [ 6490.508773] Lustre: 317275:0:(mgs_llog.c:1345:mgs_modify_param()) MGS: modify general/debug_raw_pointers=Y (mode = 0) failed: rc = -17 [ 6492.775283] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 6492.904255] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 6492.907714] Lustre: Skipped 1 previous similar message [ 6493.923848] LustreError: 317632:0:(ldlm_lib.c:1132:target_handle_connect()) lustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 6493.929870] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:36 to 0x280000401:129) [ 6494.904819] Lustre: DEBUG MARKER: oleg158-server.virtnet: executing set_default_debug -1 all [ 6497.962177] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 6499.830417] Lustre: lustre-OST0001: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x2c0000401:3 to 0x2c0000401:97) [ 6499.946775] Lustre: DEBUG MARKER: oleg158-server.virtnet: executing set_default_debug -1 all [ 6503.177948] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 6504.406100] Lustre: 319119:0:(mgs_llog.c:1345:mgs_modify_param()) MGS: modify general/lod.*.mdt_hash=crush (mode = 0) failed: rc = -17 [ 6513.636905] Lustre: lustre-MDT0000-lwp-OST0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 6513.638436] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 6513.638881] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 6513.638890] Lustre: Skipped 1 previous similar message [ 6513.641934] Lustre: Skipped 5 previous similar messages [ 6516.480166] LustreError: 319224:0:(obd_class.h:479:obd_check_dev()) Device 10 not setup [ 6516.483445] LustreError: 319224:0:(obd_class.h:479:obd_check_dev()) Skipped 5 previous similar messages [ 6516.517130] Lustre: server umount lustre-MDT0000 complete [ 6517.850532] LustreError: 317277:0:(ldlm_lockd.c:2607:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1758743504 with bad export cookie 17590846157992813910 [ 6517.854893] LustreError: MGC192.168.201.158@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 6517.855967] LustreError: 317277:0:(ldlm_lockd.c:2607:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 6518.002107] Lustre: server umount lustre-MDT0001 complete [ 6529.462218] LustreError: 319627:0:(obd_class.h:479:obd_check_dev()) Device 23 not setup [ 6529.463983] LustreError: 319627:0:(obd_class.h:479:obd_check_dev()) Skipped 15 previous similar messages [ 6529.476633] Lustre: server umount lustre-OST0000 complete [ 6539.885448] Lustre: server umount lustre-OST0001 complete [ 6541.042851] Lustre: DEBUG MARKER: SKIP: conf-sanity test_114 skipping SLOW test 114 [ 6541.575652] Lustre: DEBUG MARKER: SKIP: conf-sanity test_115 skipping excluded test 115 [ 6542.163540] Lustre: DEBUG MARKER: == conf-sanity test 116: big size MDT support ============ 15:52:08 (1758743528) [ 6549.659423] Lustre: DEBUG MARKER: oleg158-server.virtnet: executing load_modules_local [ 6552.718849] loop0: detected capacity change from 0 to 1099511627776 [ 6552.865381] SGI XFS with ACLs, security attributes, quota, no debug enabled [ 6552.868496] *** VALIDATE xfs *** [ 6552.872048] XFS (loop0): Mounting V5 Filesystem [ 6552.879906] XFS (loop0): Ending clean mount [ 6553.296111] loop1: detected capacity change from 0 to 18691697672192 [ 6567.871855] LDISKFS-fs (loop1): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 6573.615515] Lustre: DEBUG MARKER: oleg158-server.virtnet: executing set_hostid [ 6576.631203] Lustre: DEBUG MARKER: oleg158-server.virtnet: executing load_modules_local [ 6579.985228] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 6582.177990] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 6584.152026] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 6586.176830] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 6588.479292] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 6588.502137] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 6588.583615] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 6588.597118] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 6588.631201] Lustre: lustre-MDT0000: new disk, initializing [ 6588.655163] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 6588.657761] Lustre: Skipped 1 previous similar message [ 6588.663743] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 6589.957990] Lustre: DEBUG MARKER: oleg158-server.virtnet: executing set_default_debug -1 all [ 6594.070845] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 6594.097164] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 6594.130823] Lustre: 324195:0:(mgs_llog.c:1345:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 6594.157129] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [ 6594.158773] Lustre: Skipped 1 previous similar message [ 6594.197498] Lustre: lustre-MDT0001: new disk, initializing [ 6594.247122] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [ 6594.250257] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 6595.545269] Lustre: DEBUG MARKER: oleg158-server.virtnet: executing set_default_debug -1 all [ 6598.562658] Lustre: DEBUG MARKER: oleg158-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 6599.738501] Lustre: DEBUG MARKER: oleg158-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 6602.007871] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 6602.031761] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 6602.134588] Lustre: lustre-OST0000: new disk, initializing [ 6602.136639] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 6604.092311] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [ 6604.096653] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 6604.130852] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 6604.140572] Lustre: DEBUG MARKER: oleg158-server.virtnet: executing set_default_debug -1 all [ 6607.310400] Lustre: DEBUG MARKER: oleg158-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 6608.931471] Lustre: DEBUG MARKER: oleg158-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 [ 6609.020993] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [ 6610.330891] Lustre: DEBUG MARKER: oleg158-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid 50 [ 6610.402251] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [ 6613.103119] LustreError: 327136:0:(obd_class.h:479:obd_check_dev()) Device 19 not setup [ 6613.105307] LustreError: 327136:0:(obd_class.h:479:obd_check_dev()) Skipped 9 previous similar messages [ 6613.182786] Lustre: server umount lustre-OST0000 complete [ 6614.497977] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 6614.498593] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 6614.502711] Lustre: Skipped 1 previous similar message [ 6614.505798] Lustre: Skipped 2 previous similar messages [ 6619.617877] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 6619.621425] Lustre: Skipped 2 previous similar messages [ 6624.739453] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 6624.741191] Lustre: Skipped 1 previous similar message [ 6624.767744] Lustre: server umount lustre-MDT0000 complete [ 6626.006126] LustreError: 324186:0:(ldlm_lockd.c:2607:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1758743612 with bad export cookie 17590846157992815464 [ 6626.008895] LustreError: MGC192.168.201.158@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 6626.009757] LustreError: 324186:0:(ldlm_lockd.c:2607:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 6636.585676] Lustre: server umount lustre-MDT0001 complete [ 6639.215092] XFS (loop0): Unmounting Filesystem [ 6640.754350] Lustre: DEBUG MARKER: == conf-sanity test 117: lctl get_param return errors properly ========================================================== 15:53:47 (1758743627) [ 6642.982321] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 6643.123416] LustreError: 328355:0:(ldlm_lib.c:1132:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 6643.146386] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 6643.148672] Lustre: Skipped 2 previous similar messages [ 6644.398760] Lustre: DEBUG MARKER: oleg158-server.virtnet: executing set_default_debug -1 all [ 6647.107121] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 6648.514014] Lustre: DEBUG MARKER: oleg158-server.virtnet: executing set_default_debug -1 all [ 6650.177245] Lustre: DEBUG MARKER: oleg158-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 6651.291499] Lustre: DEBUG MARKER: oleg158-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 6653.440636] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 6655.235755] Lustre: DEBUG MARKER: oleg158-server.virtnet: executing set_default_debug -1 all [ 6656.825848] Lustre: DEBUG MARKER: oleg158-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 6678.502493] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 6678.502678] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 6678.505050] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 6678.508845] Lustre: Skipped 1 previous similar message [ 6678.512844] Lustre: Skipped 1 previous similar message [ 6681.711085] LustreError: 330718:0:(obd_class.h:479:obd_check_dev()) Device 21 not setup [ 6681.713103] LustreError: 330718:0:(obd_class.h:479:obd_check_dev()) Skipped 17 previous similar messages [ 6681.781780] Lustre: server umount lustre-OST0000 complete [ 6683.104970] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 6683.109360] LustreError: Skipped 1 previous similar message [ 6683.111931] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 6689.238158] Lustre: server umount lustre-MDT0000 complete [ 6690.731766] LustreError: 329075:0:(ldlm_lockd.c:2607:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1758743676 with bad export cookie 17590846157992816346 [ 6690.734244] LustreError: MGC192.168.201.158@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 6690.735759] LustreError: 329075:0:(ldlm_lockd.c:2607:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 6690.928663] Lustre: server umount lustre-MDT0001 complete [ 6697.592569] Lustre: DEBUG MARKER: oleg158-server.virtnet: executing unload_modules_local [ 6700.344518] Key type lgssc unregistered [ 6700.753955] LNet: 331656:0:(lib-ptl.c:966:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 6700.763538] LNetError: 331656:0:(acceptor.c:263:lnet_acceptor_remove_socket()) Interface ens2 not found [ 6700.771750] LNet: Removed LNI 192.168.201.158@tcp [ 6701.940199] Key type .llcrypt unregistered [ 6701.942056] Key type ._llcrypt unregistered [ 6729.508819] Key type ._llcrypt registered [ 6729.510489] Key type .llcrypt registered [ 6729.599630] Lustre: DEBUG MARKER: oleg158-server.virtnet: executing load_modules_local [ 6730.799685] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 6730.847904] alg: No test for adler32 (adler32-zlib) [ 6731.926327] Lustre: Lustre: Build Version: 2.16.58_53_g2e3f3ed [ 6732.125385] LNet: Added LNI 192.168.201.158@tcp [8/256/0/180] [ 6733.768436] Key type lgssc registered [ 6734.585154] Lustre: Echo OBD driver; http://www.lustre.org/ [ 6738.112078] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 6738.125597] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 6739.349828] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 6740.938505] Lustre: DEBUG MARKER: oleg158-server.virtnet: executing set_default_debug -1 all [ 6744.446673] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 6744.664442] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 6746.469170] Lustre: DEBUG MARKER: oleg158-server.virtnet: executing set_default_debug -1 all [ 6749.531300] Lustre: DEBUG MARKER: oleg158-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 6752.058315] Lustre: DEBUG MARKER: oleg158-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 6759.342969] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 6759.827846] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 6768.917722] Lustre: DEBUG MARKER: oleg158-server.virtnet: executing set_default_debug -1 all [ 6777.064115] Lustre: DEBUG MARKER: oleg158-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 6784.999885] LustreError: lustre-MDT0001-osp-MDT0000: operation mds_statfs to node 0@lo failed: rc = -107 [ 6785.011615] Lustre: lustre-MDT0001-osp-MDT0000: Connection to lustre-MDT0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 6785.030733] Lustre: lustre-MDT0001: Not available for connect from 0@lo (stopping) [ 6785.137246] LustreError: 335499:0:(obd_class.h:479:obd_check_dev()) Device 16 not setup [ 6785.239280] Lustre: server umount lustre-MDT0001 complete [ 6788.415199] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 6789.601937] LustreError: 333201:0:(ldlm_lib.c:1132:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 6789.606729] Lustre: lustre-MDT0001-lwp-OST0000: Connection to lustre-MDT0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 6789.641091] LustreError: 333201:0:(ldlm_lib.c:1132:target_handle_connect()) Skipped 2 previous similar messages [ 6790.212996] LustreError: 335473:0:(ldlm_lib.c:1132:target_handle_connect()) lustre-MDT0001: not available for connect from 192.168.201.58@tcp (no target). If you are running an HA pair check that the target is mounted on the other server. [ 6790.239525] LustreError: 335473:0:(ldlm_lib.c:1132:target_handle_connect()) Skipped 1 previous similar message [ 6794.734420] LustreError: 335473:0:(ldlm_lib.c:1132:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 6794.751934] LustreError: 335473:0:(ldlm_lib.c:1132:target_handle_connect()) Skipped 1 previous similar message [ 6797.341440] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 6797.502875] Lustre: MGS: Regenerating lustre-MDT0001 log by user request: rc = 0 [ 6797.505570] Lustre: Found index 1 for lustre-MDT0001, updating log [ 6797.560035] Lustre: 333192:0:(mgs_llog.c:1345:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 6798.032804] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 6802.864279] Lustre: DEBUG MARKER: oleg158-server.virtnet: executing set_default_debug -1 all [ 6803.438506] Lustre: lustre-MDT0001-lwp-OST0000: Connection restored to 0@lo (at 0@lo) [ 6805.488675] LustreError: 336433:0:(obd_mount.c:188:lustre_start_simple()) lustre-MDT0001-lwp-OST0000 attach error -17 [ 6805.501313] LustreError: 336433:0:(tgt_mount.c:669:lustre_lwp_setup()) lustre-MDT0001-lwp-OST0000: setup up failed: rc -17 [ 6805.519788] LustreError: 333194:0:(mgc_request.c:614:do_requeue()) failed processing log: -17 [ 6805.637418] LustreError: 336440:0:(obd_config.c:1997:class_config_llog_handler()) MGC192.168.201.158@tcp: cfg command failed: rc = -17 [ 6805.655976] Lustre: cmd=cf001 0:lustre-MDT0001-osp-MDT0000 1:osp 2:lustre-MDT0000-mdtlov_UUID [ 6805.655976] [ 7107.670579] Lustre: lustre-MDT0001: Not available for connect from 192.168.201.58@tcp (stopping) [ 7107.677367] Lustre: Skipped 1 previous similar message [ 7110.631293] Lustre: lustre-MDT0001-lwp-OST0000: Connection to lustre-MDT0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 7110.633572] LustreError: lustre-MDT0001-osp-MDT0000: operation mds_statfs to node 0@lo failed: rc = -107 [ 7110.647484] Lustre: lustre-MDT0001: Not available for connect from 0@lo (stopping) [ 7110.659926] Lustre: Skipped 1 previous similar message [ 7112.768429] Lustre: lustre-MDT0001: Not available for connect from 192.168.201.58@tcp (stopping) [ 7113.435677] LustreError: 345178:0:(obd_class.h:479:obd_check_dev()) Device 16 not setup [ 7113.446181] LustreError: 345178:0:(obd_class.h:479:obd_check_dev()) Skipped 5 previous similar messages [ 7113.544786] Lustre: server umount lustre-MDT0001 complete [ 7115.762262] LustreError: 333927:0:(ldlm_lib.c:1132:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 7115.777704] LustreError: 333927:0:(ldlm_lib.c:1132:target_handle_connect()) Skipped 2 previous similar messages [ 7115.907617] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 7120.871875] LustreError: 333927:0:(ldlm_lib.c:1132:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 7120.889135] LustreError: 333927:0:(ldlm_lib.c:1132:target_handle_connect()) Skipped 2 previous similar messages [ 7122.306165] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 7122.371541] Lustre: MGS: Regenerating lustre-MDT0001 log by user request: rc = 0 [ 7122.375449] Lustre: Found index 1 for lustre-MDT0001, updating log [ 7122.414615] Lustre: 333191:0:(mgs_llog.c:1345:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 7122.754573] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 7126.420339] Lustre: DEBUG MARKER: oleg158-server.virtnet: executing set_default_debug -1 all [ 7128.044431] Lustre: lustre-MDT0001-lwp-OST0000: Connection restored to 0@lo (at 0@lo) [ 7131.646932] LustreError: 346206:0:(obd_mount.c:188:lustre_start_simple()) lustre-MDT0001-lwp-OST0000 attach error -17 [ 7131.666300] LustreError: 346206:0:(tgt_mount.c:669:lustre_lwp_setup()) lustre-MDT0001-lwp-OST0000: setup up failed: rc -17 [ 7131.669393] LustreError: 333194:0:(mgc_request.c:614:do_requeue()) failed processing log: -17 [ 7131.681611] LustreError: 333194:0:(mgc_request.c:614:do_requeue()) Skipped 1 previous similar message [ 7131.721795] LustreError: 346209:0:(obd_config.c:689:class_setup()) Device 8 already setup (type osp) [ 7131.726031] LustreError: 346209:0:(obd_config.c:1997:class_config_llog_handler()) MGC192.168.201.158@tcp: cfg command failed: rc = -17 [ 7131.734771] Lustre: cmd=cf003 0:lustre-MDT0001-osp-MDT0000 1:lustre-MDT0001_UUID 2:192.168.201.158@tcp [ 7131.734771] [ 7433.285707] Lustre: lustre-MDT0001: Not available for connect from 192.168.201.58@tcp (stopping) [ 7435.236180] Lustre: lustre-MDT0001-lwp-OST0000: Connection to lustre-MDT0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 7435.237497] LustreError: lustre-MDT0001-osp-MDT0000: operation mds_statfs to node 0@lo failed: rc = -107 [ 7435.258176] Lustre: Skipped 2 previous similar messages [ 7436.399763] LustreError: 355394:0:(obd_class.h:479:obd_check_dev()) Device 16 not setup [ 7436.410305] LustreError: 355394:0:(obd_class.h:479:obd_check_dev()) Skipped 5 previous similar messages [ 7436.500639] Lustre: server umount lustre-MDT0001 complete [ 7438.403754] LustreError: 333197:0:(ldlm_lib.c:1132:target_handle_connect()) lustre-MDT0001: not available for connect from 192.168.201.58@tcp (no target). If you are running an HA pair check that the target is mounted on the other server. [ 7438.504262] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 7444.117228] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 7444.174238] Lustre: MGS: Regenerating lustre-MDT0001 log by user request: rc = 0 [ 7444.176330] Lustre: Found index 1 for lustre-MDT0001, updating log [ 7444.201935] Lustre: 333190:0:(mgs_llog.c:1345:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 7444.449727] Lustre: lustre-MDT0001: Imperative Recovery enabled, recovery window shrunk from 60-180 down to 60-180 [ 7447.012643] Lustre: DEBUG MARKER: oleg158-server.virtnet: executing set_default_debug -1 all [ 7449.576719] Lustre: lustre-MDT0001-lwp-OST0000: Connection restored to 0@lo (at 0@lo) [ 7453.673045] LustreError: 356483:0:(obd_mount.c:188:lustre_start_simple()) lustre-MDT0001-lwp-OST0000 attach error -17 [ 7453.677035] LustreError: 356483:0:(tgt_mount.c:669:lustre_lwp_setup()) lustre-MDT0001-lwp-OST0000: setup up failed: rc -17 [ 7453.680966] LustreError: 333194:0:(mgc_request.c:614:do_requeue()) failed processing log: -17 [ 7453.683608] LustreError: 333194:0:(mgc_request.c:614:do_requeue()) Skipped 1 previous similar message [ 7453.722156] LustreError: 356487:0:(lod_lov.c:207:lod_add_device()) lustre-MDT0001-osp-MDT0000: cannot connect to next dev lustre-MDT0001_UUID (-114) [ 7453.737509] LustreError: 356487:0:(obd_config.c:1997:class_config_llog_handler()) MGC192.168.201.158@tcp: cfg command failed: rc = -114 [ 7453.743920] Lustre: cmd=cf014 0:lustre-MDT0000-mdtlov 1:lustre-MDT0001_UUID 2:1 3:1 [ 7453.743920] [ 7767.012518] Lustre: lustre-MDT0000-lwp-OST0000: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 7767.013530] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 7767.014265] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 7767.014270] Lustre: Skipped 2 previous similar messages [ 7767.020107] Lustre: Skipped 1 previous similar message [ 7769.352582] LustreError: 366534:0:(obd_class.h:479:obd_check_dev()) Device 9 not setup [ 7769.356624] LustreError: 366534:0:(obd_class.h:479:obd_check_dev()) Skipped 5 previous similar messages [ 7769.401602] Lustre: server umount lustre-MDT0000 complete [ 7771.146914] LustreError: 333921:0:(ldlm_lockd.c:2607:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1758744757 with bad export cookie 7147800713699774089 [ 7771.151160] LustreError: MGC192.168.201.158@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 7771.152106] LustreError: 333921:0:(ldlm_lockd.c:2607:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 7772.130737] LustreError: 333197:0:(ldlm_lib.c:1132:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 7772.140511] LustreError: 333197:0:(ldlm_lib.c:1132:target_handle_connect()) Skipped 3 previous similar messages [ 7777.286574] LustreError: 366736:0:(obd_class.h:479:obd_check_dev()) Device 16 not setup [ 7777.290205] LustreError: 366736:0:(obd_class.h:479:obd_check_dev()) Skipped 5 previous similar messages [ 7777.415558] Lustre: server umount lustre-MDT0001 complete [ 7779.450965] Lustre: server umount lustre-OST0000 complete [ 7784.005254] Lustre: DEBUG MARKER: oleg158-server.virtnet: executing set_hostid [ 7788.838951] Lustre: DEBUG MARKER: oleg158-server.virtnet: executing load_modules_local [ 7793.586723] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 7797.100592] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 7800.482533] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 7803.807126] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 7807.414952] Lustre: DEBUG MARKER: == conf-sanity test 120: cross-target rename should not create bad symlinks ========================================================== 16:13:13 (1758744793) [ 7810.961992] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 7811.005548] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 7811.155267] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 7811.181670] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 7811.241646] Lustre: lustre-MDT0000: new disk, initializing [ 7811.307233] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 7811.322547] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 7813.674713] Lustre: DEBUG MARKER: oleg158-server.virtnet: executing set_default_debug -1 all [ 7819.980654] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 7820.016076] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 7820.077497] Lustre: 369469:0:(mgs_llog.c:1345:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 7820.118632] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [ 7820.121268] Lustre: Skipped 1 previous similar message [ 7820.170669] Lustre: lustre-MDT0001: new disk, initializing [ 7820.232991] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 7820.253897] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [ 7820.258827] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 7822.057403] Lustre: DEBUG MARKER: oleg158-server.virtnet: executing set_default_debug -1 all [ 7826.000122] Lustre: DEBUG MARKER: oleg158-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 7827.645749] Lustre: DEBUG MARKER: oleg158-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 7830.738419] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 7830.778515] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 7830.894969] Lustre: lustre-OST0000: new disk, initializing [ 7830.897302] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 7830.934299] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 7832.469056] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [ 7832.478706] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 7832.543483] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 7833.721300] Lustre: DEBUG MARKER: oleg158-server.virtnet: executing set_default_debug -1 all [ 7837.767944] Lustre: DEBUG MARKER: oleg158-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 7842.787207] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 7842.788962] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 7842.792576] Lustre: Skipped 4 previous similar messages [ 7842.799754] Lustre: Skipped 6 previous similar messages [ 7847.246465] LustreError: 371911:0:(obd_class.h:479:obd_check_dev()) Device 22 not setup [ 7847.255242] LustreError: 371911:0:(obd_class.h:479:obd_check_dev()) Skipped 11 previous similar messages [ 7847.364033] Lustre: server umount lustre-MDT0000 complete [ 7847.907191] LustreError: 369481:0:(ldlm_lib.c:1132:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 7847.917986] LustreError: 369481:0:(ldlm_lib.c:1132:target_handle_connect()) Skipped 3 previous similar messages [ 7849.217919] LustreError: 369461:0:(ldlm_lockd.c:2607:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1758744835 with bad export cookie 7147800713699776056 [ 7849.219844] LustreError: 370262:0:(osp_object.c:617:osp_attr_get()) lustre-MDT0000-osp-MDT0001: osp_attr_get update error [0x200000401:0x2:0x0]: rc = -5 [ 7849.220094] LustreError: MGC192.168.201.158@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 7849.226460] LustreError: 369461:0:(ldlm_lockd.c:2607:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 7849.231409] LustreError: 370262:0:(client.c:1370:ptlrpc_import_delay_req()) @@@ IMP_CLOSED req@00000000354aba8f x1844176453220608/t0(0) o1000->lustre-MDT0000-osp-MDT0001@0@lo:24/4 lens 264/4320 e 0 to 0 dl 0 ref 2 fl Rpc:QU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 [ 7849.413814] Lustre: server umount lustre-MDT0001 complete [ 7861.531422] Lustre: server umount lustre-OST0000 complete [ 7875.250595] Lustre: DEBUG MARKER: oleg158-server.virtnet: executing set_hostid [ 7880.027734] Lustre: DEBUG MARKER: oleg158-server.virtnet: executing load_modules_local [ 7885.251682] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 7888.984029] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 7891.718816] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 7894.774688] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 7895.669244] Lustre: DEBUG MARKER: == conf-sanity test complete, duration 7685 sec ========== 16:14:42 (1758744882) [ 7896.495670] Lustre: DEBUG MARKER: === conf-sanity: start cleanup 16:14:43 (1758744883) === [ 7898.078775] Lustre: DEBUG MARKER: === conf-sanity: finish cleanup 16:14:44 (1758744884) === [ 7916.409596] Lustre: DEBUG MARKER: oleg158-server.virtnet: executing unload_modules_local [ 7917.743412] Key type lgssc unregistered [ 7917.900681] LNet: 377416:0:(lib-ptl.c:966:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 7917.904782] LNetError: 377416:0:(acceptor.c:263:lnet_acceptor_remove_socket()) Interface ens2 not found [ 7917.912605] LNet: Removed LNI 192.168.201.158@tcp [ 7918.319684] Key type .llcrypt unregistered [ 7918.321388] Key type ._llcrypt unregistered