[ 0.000000] Linux version 4.18.0rh8.10-debug (green@maintenance) (gcc version 8.5.0 20210514 (Red Hat 8.5.0-26) (GCC)) #2 SMP Mon Jul 14 01:24:22 EDT 2025 [ 0.000000] Command line: rd.shell root=nbd:192.168.200.253:rocky8.10:ext4:ro:-p,-b4096 ro crashkernel=256M panic=1 nomodeset ipmtu=9000 ip=dhcp rd.neednet=1 init_on_free=off mitigations=off console=ttyS1,115200 audit=0 [ 0.000000] x86/fpu: Supporting XSAVE feature 0x001: 'x87 floating point registers' [ 0.000000] x86/fpu: Supporting XSAVE feature 0x002: 'SSE registers' [ 0.000000] x86/fpu: Supporting XSAVE feature 0x004: 'AVX registers' [ 0.000000] x86/fpu: xstate_offset[2]: 576, xstate_sizes[2]: 256 [ 0.000000] x86/fpu: Enabled xstate features 0x7, context size is 832 bytes, using 'standard' format. [ 0.000000] signal: max sigframe size: 1776 [ 0.000000] BIOS-provided physical RAM map: [ 0.000000] BIOS-e820: [mem 0x0000000000000000-0x000000000009fbff] usable [ 0.000000] BIOS-e820: [mem 0x000000000009fc00-0x000000000009ffff] reserved [ 0.000000] BIOS-e820: [mem 0x00000000000f0000-0x00000000000fffff] reserved [ 0.000000] BIOS-e820: [mem 0x0000000000100000-0x00000000bffcdfff] usable [ 0.000000] BIOS-e820: [mem 0x00000000bffce000-0x00000000bfffffff] reserved [ 0.000000] BIOS-e820: [mem 0x00000000feffc000-0x00000000feffffff] reserved [ 0.000000] BIOS-e820: [mem 0x00000000fffc0000-0x00000000ffffffff] reserved [ 0.000000] BIOS-e820: [mem 0x0000000100000000-0x0000000146dfffff] usable [ 0.000000] NX (Execute Disable) protection: active [ 0.000000] SMBIOS 2.8 present. [ 0.000000] DMI: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.17.0-10.fc44 06/10/2025 [ 0.000000] Hypervisor detected: KVM [ 0.000000] kvm-clock: Using msrs 4b564d01 and 4b564d00 [ 0.000000] kvm-clock: using sched offset of 505620318 cycles [ 0.000000] clocksource: kvm-clock: mask: 0xffffffffffffffff max_cycles: 0x1cd42e4dffb, max_idle_ns: 881590591483 ns [ 0.000000] tsc: Detected 2399.998 MHz processor [ 0.000000] last_pfn = 0x146e00 max_arch_pfn = 0x400000000 [ 0.000000] x86/PAT: Configuration [0-7]: WB WC UC- UC WB WP UC- WT [ 0.000000] last_pfn = 0xbffce max_arch_pfn = 0x400000000 [ 0.000000] found SMP MP-table at [mem 0x000f54b0-0x000f54bf] [ 0.000000] RAMDISK: [mem 0xbcc54000-0xbffbffff] [ 0.000000] ACPI: Early table checksum verification disabled [ 0.000000] ACPI: RSDP 0x00000000000F52D0 000014 (v00 BOCHS ) [ 0.000000] ACPI: RSDT 0x00000000BFFE247C 000034 (v01 BOCHS BXPC 00000001 BXPC 00000001) [ 0.000000] ACPI: FACP 0x00000000BFFE2318 000074 (v01 BOCHS BXPC 00000001 BXPC 00000001) [ 0.000000] ACPI: DSDT 0x00000000BFFE0040 0022D8 (v01 BOCHS BXPC 00000001 BXPC 00000001) [ 0.000000] ACPI: FACS 0x00000000BFFE0000 000040 [ 0.000000] ACPI: APIC 0x00000000BFFE238C 000090 (v03 BOCHS BXPC 00000001 BXPC 00000001) [ 0.000000] ACPI: HPET 0x00000000BFFE241C 000038 (v01 BOCHS BXPC 00000001 BXPC 00000001) [ 0.000000] ACPI: WAET 0x00000000BFFE2454 000028 (v01 BOCHS BXPC 00000001 BXPC 00000001) [ 0.000000] ACPI: Reserving FACP table memory at [mem 0xbffe2318-0xbffe238b] [ 0.000000] ACPI: Reserving DSDT table memory at [mem 0xbffe0040-0xbffe2317] [ 0.000000] ACPI: Reserving FACS table memory at [mem 0xbffe0000-0xbffe003f] [ 0.000000] ACPI: Reserving APIC table memory at [mem 0xbffe238c-0xbffe241b] [ 0.000000] ACPI: Reserving HPET table memory at [mem 0xbffe241c-0xbffe2453] [ 0.000000] ACPI: Reserving WAET table memory at [mem 0xbffe2454-0xbffe247b] [ 0.000000] No NUMA configuration found [ 0.000000] Faking a node at [mem 0x0000000000000000-0x0000000146dfffff] [ 0.000000] NODE_DATA(0) allocated [mem 0x1465a3000-0x1465cdfff] [ 0.000000] Reserving 256MB of memory at 2752MB for crashkernel (System RAM: 4205MB) [ 0.000000] Zone ranges: [ 0.000000] DMA [mem 0x0000000000001000-0x0000000000ffffff] [ 0.000000] DMA32 [mem 0x0000000001000000-0x00000000ffffffff] [ 0.000000] Normal [mem 0x0000000100000000-0x0000000146dfffff] [ 0.000000] Device empty [ 0.000000] Movable zone start for each node [ 0.000000] Early memory node ranges [ 0.000000] node 0: [mem 0x0000000000001000-0x000000000009efff] [ 0.000000] node 0: [mem 0x0000000000100000-0x00000000bffcdfff] [ 0.000000] node 0: [mem 0x0000000100000000-0x0000000146dfffff] [ 0.000000] Zeroed struct page in unavailable ranges: 4756 pages [ 0.000000] Initmem setup node 0 [mem 0x0000000000001000-0x0000000146dfffff] [ 0.000000] ACPI: PM-Timer IO Port: 0x608 [ 0.000000] ACPI: LAPIC_NMI (acpi_id[0xff] dfl dfl lint[0x1]) [ 0.000000] IOAPIC[0]: apic_id 0, version 17, address 0xfec00000, GSI 0-23 [ 0.000000] ACPI: INT_SRC_OVR (bus 0 bus_irq 0 global_irq 2 dfl dfl) [ 0.000000] ACPI: INT_SRC_OVR (bus 0 bus_irq 5 global_irq 5 high level) [ 0.000000] ACPI: INT_SRC_OVR (bus 0 bus_irq 9 global_irq 9 high level) [ 0.000000] ACPI: INT_SRC_OVR (bus 0 bus_irq 10 global_irq 10 high level) [ 0.000000] ACPI: INT_SRC_OVR (bus 0 bus_irq 11 global_irq 11 high level) [ 0.000000] Using ACPI (MADT) for SMP configuration information [ 0.000000] ACPI: HPET id: 0x8086a201 base: 0xfed00000 [ 0.000000] TSC deadline timer available [ 0.000000] smpboot: Allowing 4 CPUs, 0 hotplug CPUs [ 0.000000] kvm-guest: KVM setup pv remote TLB flush [ 0.000000] kvm-guest: setup PV sched yield [ 0.000000] PM: Registered nosave memory: [mem 0x00000000-0x00000fff] [ 0.000000] PM: Registered nosave memory: [mem 0x0009f000-0x0009ffff] [ 0.000000] PM: Registered nosave memory: [mem 0x000a0000-0x000effff] [ 0.000000] PM: Registered nosave memory: [mem 0x000f0000-0x000fffff] [ 0.000000] PM: Registered nosave memory: [mem 0xbffce000-0xbfffffff] [ 0.000000] PM: Registered nosave memory: [mem 0xc0000000-0xfeffbfff] [ 0.000000] PM: Registered nosave memory: [mem 0xfeffc000-0xfeffffff] [ 0.000000] PM: Registered nosave memory: [mem 0xff000000-0xfffbffff] [ 0.000000] PM: Registered nosave memory: [mem 0xfffc0000-0xffffffff] [ 0.000000] [mem 0xc0000000-0xfeffbfff] available for PCI devices [ 0.000000] Booting paravirtualized kernel on KVM [ 0.000000] clocksource: refined-jiffies: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 1910969940391419 ns [ 0.000000] setup_percpu: NR_CPUS:8192 nr_cpumask_bits:4 nr_cpu_ids:4 nr_node_ids:1 [ 0.000000] percpu: Embedded 63 pages/cpu s221184 r8192 d28672 u524288 [ 0.000000] kvm-guest: PV spinlocks enabled [ 0.000000] PV qspinlock hash table entries: 256 (order: 0, 4096 bytes, linear) [ 0.000000] Built 1 zonelists, mobility grouping on. Total pages: 1059606 [ 0.000000] Policy zone: Normal [ 0.000000] Kernel command line: rd.shell root=nbd:192.168.200.253:rocky8.10:ext4:ro:-p,-b4096 ro crashkernel=256M panic=1 nomodeset ipmtu=9000 ip=dhcp rd.neednet=1 init_on_free=off mitigations=off console=ttyS1,115200 audit=0 [ 0.000000] Specific versions of hardware are certified with Red Hat Enterprise Linux 8. Please see the list of hardware certified with Red Hat Enterprise Linux 8 at https://catalog.redhat.com. [ 0.000000] audit: disabled (until reboot) [ 0.000000] software IO TLB: area num 4. [ 0.000000] Memory: 2829652K/4306352K available (18435K kernel code, 11221K rwdata, 7248K rodata, 2908K init, 18040K bss, 524584K reserved, 0K cma-reserved) [ 0.000000] SLUB: HWalign=64, Order=0-3, MinObjects=0, CPUs=4, Nodes=1 [ 0.000000] kmemleak: Kernel memory leak detector disabled [ 0.000000] ftrace: allocating 41240 entries in 162 pages [ 0.000000] ftrace: allocated 162 pages with 3 groups [ 0.000000] rcu: Hierarchical RCU implementation. [ 0.000000] rcu: RCU event tracing is enabled. [ 0.000000] rcu: RCU restricting CPUs from NR_CPUS=8192 to nr_cpu_ids=4. [ 0.000000] rcu: RCU callback double-/use-after-free debug enabled. [ 0.000000] Rude variant of Tasks RCU enabled. [ 0.000000] Tracing variant of Tasks RCU enabled. [ 0.000000] rcu: RCU calculated value of scheduler-enlistment delay is 100 jiffies. [ 0.000000] rcu: Adjusting geometry for rcu_fanout_leaf=16, nr_cpu_ids=4 [ 0.000000] NR_IRQS: 524544, nr_irqs: 456, preallocated irqs: 16 [ 0.000000] random: get_random_bytes called from start_kernel+0x622/0x9a8 with crng_init=0 [ 0.001000] Console: colour *CGA 80x25 [ 0.001000] printk: console [ttyS1] enabled [ 0.001000] ACPI: Core revision 20220331 [ 0.001000] clocksource: hpet: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 19112604467 ns [ 0.001011] APIC: Switch to symmetric I/O mode setup [ 0.003132] x2apic enabled [ 0.004011] Switched APIC routing to physical x2apic. [ 0.005016] kvm-guest: setup PV IPIs [ 0.008000] ..TIMER: vector=0x30 apic1=0 pin1=2 apic2=-1 pin2=-1 [ 0.008000] clocksource: tsc-early: mask: 0xffffffffffffffff max_cycles: 0x229835b7123, max_idle_ns: 440795242976 ns [ 0.008026] Calibrating delay loop (skipped) preset value.. 4799.99 BogoMIPS (lpj=2399998) [ 0.009013] pid_max: default: 32768 minimum: 301 [ 0.010146] LSM: Security Framework initializing [ 0.012070] Yama: becoming mindful. [ 0.013057] SELinux: Initializing. [ 0.014080] *** VALIDATE selinux *** [ 0.022889] Dentry cache hash table entries: 1048576 (order: 11, 8388608 bytes, vmalloc) [ 0.027811] Inode-cache hash table entries: 524288 (order: 10, 4194304 bytes, vmalloc) [ 0.028157] Mount-cache hash table entries: 16384 (order: 5, 131072 bytes, vmalloc) [ 0.029145] Mountpoint-cache hash table entries: 16384 (order: 5, 131072 bytes, vmalloc) [ 0.030142] *** VALIDATE tmpfs *** [ 0.032384] *** VALIDATE proc *** [ 0.033294] *** VALIDATE cgroup *** [ 0.034009] *** VALIDATE cgroup2 *** [ 0.035325] x86/cpu: User Mode Instruction Prevention (UMIP) activated [ 0.037005] Last level iTLB entries: 4KB 0, 2MB 0, 4MB 0 [ 0.038013] Last level dTLB entries: 4KB 0, 2MB 0, 4MB 0, 1GB 0 [ 0.039031] Spectre V2 : User space: Vulnerable [ 0.040013] Speculative Store Bypass: Vulnerable [ 0.043885] debug: unmapping init [mem 0xffffffffa8a59000-0xffffffffa8a60fff] [ 0.045945] smpboot: CPU0: Intel(R) Xeon(R) CPU E5-2695 v2 @ 2.40GHz (family: 0x6, model: 0x3e, stepping: 0x4) [ 0.046789] Performance Events: IvyBridge events, full-width counters, Intel PMU driver. [ 0.047028] ... version: 2 [ 0.048010] ... bit width: 48 [ 0.049010] ... generic registers: 4 [ 0.050018] ... value mask: 0000ffffffffffff [ 0.051021] ... max period: 00007fffffffffff [ 0.052016] ... fixed-purpose events: 3 [ 0.053019] ... event mask: 000000070000000f [ 0.054331] rcu: Hierarchical SRCU implementation. [ 0.056565] smp: Bringing up secondary CPUs ... [ 0.057625] x86: Booting SMP configuration: [ 0.058030] .... node #0, CPUs: #1 #2 #3 [ 0.061627] smp: Brought up 1 node, 4 CPUs [ 0.063016] smpboot: Max logical packages: 1 [ 0.064024] smpboot: Total of 4 processors activated (19199.98 BogoMIPS) [ 0.144127] node 0 deferred pages initialised in 77ms [ 0.148136] devtmpfs: initialized [ 0.149235] x86/mm: Memory block size: 128MB [ 0.151561] gcov: version magic: 0x41383552 [ 0.154266] clocksource: jiffies: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 1911260446275000 ns [ 0.156099] futex hash table entries: 1024 (order: 4, 65536 bytes, vmalloc) [ 0.158344] pinctrl core: initialized pinctrl subsystem [ 0.159186] [ 0.159483] ************************************************************* [ 0.162015] ** NOTICE NOTICE NOTICE NOTICE NOTICE NOTICE NOTICE ** [ 0.163013] ** ** [ 0.165015] ** IOMMU DebugFS SUPPORT HAS BEEN ENABLED IN THIS KERNEL ** [ 0.167019] ** ** [ 0.169011] ** This means that this kernel is built to expose internal ** [ 0.170008] ** IOMMU data structures, which may compromise security on ** [ 0.172017] ** your system. ** [ 0.174015] ** ** [ 0.177016] ** If you see this message and you are not debugging the ** [ 0.178012] ** kernel, report this immediately to your vendor! ** [ 0.180026] ** ** [ 0.183015] ** NOTICE NOTICE NOTICE NOTICE NOTICE NOTICE NOTICE ** [ 0.184013] ************************************************************* [ 0.186960] NET: Registered protocol family 16 [ 0.189464] DMA: preallocated 512 KiB GFP_KERNEL pool for atomic allocations [ 0.192073] DMA: preallocated 512 KiB GFP_KERNEL|GFP_DMA pool for atomic allocations [ 0.195081] DMA: preallocated 512 KiB GFP_KERNEL|GFP_DMA32 pool for atomic allocations [ 0.199100] cpuidle: using governor menu [ 0.200849] acpiphp: ACPI Hot Plug PCI Controller Driver version: 0.5 [ 0.203453] PCI: Using configuration type 1 for base access [ 0.205141] core: PMU erratum BJ122, BV98, HSD29 worked around, HT is on [ 0.215173] HugeTLB registered 1.00 GiB page size, pre-allocated 0 pages [ 0.216039] HugeTLB registered 2.00 MiB page size, pre-allocated 0 pages [ 0.217254] cryptd: max_cpu_qlen set to 1000 [ 0.221310] ACPI: Added _OSI(Module Device) [ 0.222012] ACPI: Added _OSI(Processor Device) [ 0.223009] ACPI: Added _OSI(3.0 _SCP Extensions) [ 0.223989] ACPI: Added _OSI(Processor Aggregator Device) [ 0.229286] ACPI: 1 ACPI AML tables successfully acquired and loaded [ 0.236611] ACPI: Interpreter enabled [ 0.238068] ACPI: PM: (supports S0 S3 S4 S5) [ 0.240018] ACPI: Using IOAPIC for interrupt routing [ 0.241113] PCI: Using host bridge windows from ACPI; if necessary, use "pci=nocrs" and report a bug [ 0.245435] ACPI: Enabled 2 GPEs in block 00 to 0F [ 0.256859] ACPI: PCI Root Bridge [PCI0] (domain 0000 [bus 00-ff]) [ 0.259054] acpi PNP0A03:00: _OSC: OS supports [ASPM ClockPM Segments MSI HPX-Type3] [ 0.262129] acpi PNP0A03:00: _OSC: not requesting OS control; OS requires [ExtendedConfig ASPM ClockPM MSI] [ 0.267092] acpi PNP0A03:00: fail to add MMCONFIG information, can't access extended PCI configuration space under this bridge. [ 0.272451] acpiphp: Slot [2] registered [ 0.273128] acpiphp: Slot [5] registered [ 0.274144] acpiphp: Slot [6] registered [ 0.276130] acpiphp: Slot [7] registered [ 0.277137] acpiphp: Slot [8] registered [ 0.279071] acpiphp: Slot [9] registered [ 0.280078] acpiphp: Slot [10] registered [ 0.281098] acpiphp: Slot [3] registered [ 0.282098] acpiphp: Slot [4] registered [ 0.284111] acpiphp: Slot [11] registered [ 0.285088] acpiphp: Slot [12] registered [ 0.286106] acpiphp: Slot [13] registered [ 0.288127] acpiphp: Slot [14] registered [ 0.289121] acpiphp: Slot [15] registered [ 0.291144] acpiphp: Slot [16] registered [ 0.292090] acpiphp: Slot [17] registered [ 0.293104] acpiphp: Slot [18] registered [ 0.295112] acpiphp: Slot [19] registered [ 0.297107] acpiphp: Slot [20] registered [ 0.298108] acpiphp: Slot [21] registered [ 0.299119] acpiphp: Slot [22] registered [ 0.300095] acpiphp: Slot [23] registered [ 0.302088] acpiphp: Slot [24] registered [ 0.303106] acpiphp: Slot [25] registered [ 0.305156] acpiphp: Slot [26] registered [ 0.306109] acpiphp: Slot [27] registered [ 0.308177] acpiphp: Slot [28] registered [ 0.309111] acpiphp: Slot [29] registered [ 0.311116] acpiphp: Slot [30] registered [ 0.312106] acpiphp: Slot [31] registered [ 0.314092] PCI host bridge to bus 0000:00 [ 0.315016] pci_bus 0000:00: root bus resource [io 0x0000-0x0cf7 window] [ 0.316019] pci_bus 0000:00: root bus resource [io 0x0d00-0xffff window] [ 0.319024] pci_bus 0000:00: root bus resource [mem 0x000a0000-0x000bffff window] [ 0.321023] pci_bus 0000:00: root bus resource [mem 0xc0000000-0xfebfffff window] [ 0.324028] pci_bus 0000:00: root bus resource [mem 0xe0000000000-0xe007fffffff window] [ 0.326024] pci_bus 0000:00: root bus resource [bus 00-ff] [ 0.328186] pci 0000:00:00.0: [8086:1237] type 00 class 0x060000 [ 0.331071] pci 0000:00:01.0: [8086:7000] type 00 class 0x060100 [ 0.334413] pci 0000:00:01.1: [8086:7010] type 00 class 0x010180 [ 0.343784] pci 0000:00:01.1: reg 0x20: [io 0xc320-0xc32f] [ 0.351039] pci 0000:00:01.1: legacy IDE quirk: reg 0x10: [io 0x01f0-0x01f7] [ 0.354022] pci 0000:00:01.1: legacy IDE quirk: reg 0x14: [io 0x03f6] [ 0.355013] pci 0000:00:01.1: legacy IDE quirk: reg 0x18: [io 0x0170-0x0177] [ 0.357021] pci 0000:00:01.1: legacy IDE quirk: reg 0x1c: [io 0x0376] [ 0.359457] pci 0000:00:01.3: [8086:7113] type 00 class 0x068000 [ 0.361602] pci 0000:00:01.3: quirk: [io 0x0600-0x063f] claimed by PIIX4 ACPI [ 0.364029] pci 0000:00:01.3: quirk: [io 0x0700-0x070f] claimed by PIIX4 SMB [ 0.366896] pci 0000:00:02.0: [1af4:1000] type 00 class 0x020000 [ 0.372017] pci 0000:00:02.0: reg 0x10: [io 0xc300-0xc31f] [ 0.382000] pci 0000:00:02.0: reg 0x20: [mem 0xe0000000000-0xe0000003fff 64bit pref] [ 0.386017] pci 0000:00:02.0: reg 0x30: [mem 0xfeb80000-0xfebbffff pref] [ 0.392000] pci 0000:00:05.0: [1af4:1001] type 00 class 0x010000 [ 0.401018] pci 0000:00:05.0: reg 0x10: [io 0xc000-0xc07f] [ 0.408019] pci 0000:00:05.0: reg 0x14: [mem 0xfebc0000-0xfebc0fff] [ 0.430024] pci 0000:00:05.0: reg 0x20: [mem 0xe0000004000-0xe0000007fff 64bit pref] [ 0.450316] pci 0000:00:06.0: [1af4:1001] type 00 class 0x010000 [ 0.456025] pci 0000:00:06.0: reg 0x10: [io 0xc080-0xc0ff] [ 0.461021] pci 0000:00:06.0: reg 0x14: [mem 0xfebc1000-0xfebc1fff] [ 0.477022] pci 0000:00:06.0: reg 0x20: [mem 0xe0000008000-0xe000000bfff 64bit pref] [ 0.486000] pci 0000:00:07.0: [1af4:1001] type 00 class 0x010000 [ 0.491019] pci 0000:00:07.0: reg 0x10: [io 0xc100-0xc17f] [ 0.499021] pci 0000:00:07.0: reg 0x14: [mem 0xfebc2000-0xfebc2fff] [ 0.520015] pci 0000:00:07.0: reg 0x20: [mem 0xe000000c000-0xe000000ffff 64bit pref] [ 0.530818] pci 0000:00:08.0: [1af4:1001] type 00 class 0x010000 [ 0.542018] pci 0000:00:08.0: reg 0x10: [io 0xc180-0xc1ff] [ 0.558030] pci 0000:00:08.0: reg 0x14: [mem 0xfebc3000-0xfebc3fff] [ 0.574019] pci 0000:00:08.0: reg 0x20: [mem 0xe0000010000-0xe0000013fff 64bit pref] [ 0.583510] pci 0000:00:09.0: [1af4:1001] type 00 class 0x010000 [ 0.592017] pci 0000:00:09.0: reg 0x10: [io 0xc200-0xc27f] [ 0.599017] pci 0000:00:09.0: reg 0x14: [mem 0xfebc4000-0xfebc4fff] [ 0.618023] pci 0000:00:09.0: reg 0x20: [mem 0xe0000014000-0xe0000017fff 64bit pref] [ 0.630268] pci 0000:00:0a.0: [1af4:1001] type 00 class 0x010000 [ 0.637018] pci 0000:00:0a.0: reg 0x10: [io 0xc280-0xc2ff] [ 0.648017] pci 0000:00:0a.0: reg 0x14: [mem 0xfebc5000-0xfebc5fff] [ 0.662019] pci 0000:00:0a.0: reg 0x20: [mem 0xe0000018000-0xe000001bfff 64bit pref] [ 0.673615] ACPI: PCI: Interrupt link LNKA configured for IRQ 10 [ 0.676471] ACPI: PCI: Interrupt link LNKB configured for IRQ 10 [ 0.679352] ACPI: PCI: Interrupt link LNKC configured for IRQ 11 [ 0.681769] ACPI: PCI: Interrupt link LNKD configured for IRQ 11 [ 0.684256] ACPI: PCI: Interrupt link LNKS configured for IRQ 9 [ 0.691033] iommu: Default domain type: Passthrough [ 0.693570] SCSI subsystem initialized [ 0.694127] ACPI: bus type USB registered [ 0.696112] usbcore: registered new interface driver usbfs [ 0.698093] usbcore: registered new interface driver hub [ 0.700105] usbcore: registered new device driver usb [ 0.702207] pps_core: LinuxPPS API ver. 1 registered [ 0.704013] pps_core: Software ver. 5.3.6 - Copyright 2005-2007 Rodolfo Giometti [ 0.707083] PTP clock support registered [ 0.710092] EDAC MC: Ver: 3.0.0 [ 0.712158] PCI: Using ACPI for IRQ routing [ 0.713931] NetLabel: Initializing [ 0.716014] NetLabel: domain hash size = 128 [ 0.718012] NetLabel: protocols = UNLABELED CIPSOv4 CALIPSO [ 0.720103] NetLabel: unlabeled traffic allowed by default [ 0.722119] vgaarb: loaded [ 0.724357] hpet0: at MMIO 0xfed00000, IRQs 2, 8, 0 [ 0.726018] hpet0: 3 comparators, 64-bit 100.000000 MHz counter [ 0.732572] clocksource: Switched to clocksource kvm-clock [ 0.856972] VFS: Disk quotas dquot_6.6.0 [ 0.858269] VFS: Dquot-cache hash table entries: 512 (order 0, 4096 bytes) [ 0.861088] *** VALIDATE ramfs *** [ 0.862487] *** VALIDATE hugetlbfs *** [ 0.864545] pnp: PnP ACPI init [ 0.867498] pnp: PnP ACPI: found 6 devices [ 0.886237] clocksource: acpi_pm: mask: 0xffffff max_cycles: 0xffffff, max_idle_ns: 2085701024 ns [ 0.889460] pci_bus 0000:00: resource 4 [io 0x0000-0x0cf7 window] [ 0.892283] pci_bus 0000:00: resource 5 [io 0x0d00-0xffff window] [ 0.894417] pci_bus 0000:00: resource 6 [mem 0x000a0000-0x000bffff window] [ 0.897087] pci_bus 0000:00: resource 7 [mem 0xc0000000-0xfebfffff window] [ 0.899715] pci_bus 0000:00: resource 8 [mem 0xe0000000000-0xe007fffffff window] [ 0.902384] NET: Registered protocol family 2 [ 0.904724] IP idents hash table entries: 131072 (order: 8, 1048576 bytes, vmalloc) [ 0.909018] tcp_listen_portaddr_hash hash table entries: 4096 (order: 5, 163840 bytes, vmalloc) [ 0.912565] TCP established hash table entries: 65536 (order: 7, 524288 bytes, vmalloc) [ 0.917140] TCP bind hash table entries: 65536 (order: 9, 2097152 bytes, vmalloc) [ 0.919582] TCP: Hash tables configured (established 65536 bind 65536) [ 0.922117] MPTCP token hash table entries: 8192 (order: 6, 393216 bytes, vmalloc) [ 0.924375] UDP hash table entries: 4096 (order: 6, 393216 bytes, vmalloc) [ 0.927182] UDP-Lite hash table entries: 4096 (order: 6, 393216 bytes, vmalloc) [ 0.930883] NET: Registered protocol family 1 [ 0.934041] RPC: Registered named UNIX socket transport module. [ 0.936902] RPC: Registered udp transport module. [ 0.938864] RPC: Registered tcp transport module. [ 0.940804] RPC: Registered tcp NFSv4.1 backchannel transport module. [ 0.943461] NET: Registered protocol family 44 [ 0.944945] pci 0000:00:00.0: Limiting direct PCI/PCI transfers [ 0.948560] pci 0000:00:01.0: PIIX3: Enabling Passive Release [ 0.952100] pci 0000:00:01.0: Activating ISA DMA hang workarounds [ 0.955224] PCI: CLS 0 bytes, default 64 [ 0.956573] Unpacking initramfs... [ 2.396384] debug: unmapping init [mem 0xffffa039bcc54000-0xffffa039bffbffff] [ 2.401224] PCI-DMA: Using software bounce buffering for IO (SWIOTLB) [ 2.403588] software IO TLB: mapped [mem 0x00000000a8000000-0x00000000ac000000] (64MB) [ 2.406646] clocksource: tsc: mask: 0xffffffffffffffff max_cycles: 0x229835b7123, max_idle_ns: 440795242976 ns [ 2.929407] Initialise system trusted keyrings [ 2.931164] Key type blacklist registered [ 2.933304] workingset: timestamp_bits=36 max_order=20 bucket_order=0 [ 2.943352] zbud: loaded [ 2.946391] *** VALIDATE nfs *** [ 2.947775] *** VALIDATE nfs4 *** [ 2.949343] pstore: using deflate compression [ 2.953460] Platform Keyring initialized [ 3.056842] NET: Registered protocol family 38 [ 3.058169] Key type asymmetric registered [ 3.059916] Asymmetric key parser 'x509' registered [ 3.061774] Block layer SCSI generic (bsg) driver version 0.4 loaded (major 247) [ 3.064864] io scheduler mq-deadline registered [ 3.066616] io scheduler kyber registered [ 3.070049] io scheduler bfq registered [ 3.071356] atomic64_test: passed for x86-64 platform with CX8 and with SSE [ 3.073101] shpchp: Standard Hot Plug PCI Controller Driver version: 0.4 [ 3.075011] input: Power Button as /devices/LNXSYSTM:00/LNXPWRBN:00/input/input0 [ 3.076664] ACPI: Power Button [PWRF] [ 3.081182] ACPI: \_SB_.LNKB: Enabled at IRQ 10 [ 3.086328] ACPI: \_SB_.LNKA: Enabled at IRQ 11 [ 3.093841] ACPI: \_SB_.LNKC: Enabled at IRQ 11 [ 3.098565] ACPI: \_SB_.LNKD: Enabled at IRQ 10 [ 3.113528] Serial: 8250/16550 driver, 4 ports, IRQ sharing enabled [ 3.141526] 00:03: ttyS1 at I/O 0x2f8 (irq = 3, base_baud = 115200) is a 16550A [ 3.169683] 00:04: ttyS0 at I/O 0x3f8 (irq = 4, base_baud = 115200) is a 16550A [ 3.176117] Non-volatile memory driver v1.3 [ 3.177697] Linux agpgart interface v0.103 [ 3.211488] virtio_blk virtio1: [vda] 146552 512-byte logical blocks (75.0 MB/71.6 MiB) [ 3.214755] vda: detected capacity change from 0 to 75034624 [ 3.229311] virtio_blk virtio2: [vdb] 2097152 512-byte logical blocks (1.07 GB/1.00 GiB) [ 3.232460] vdb: detected capacity change from 0 to 1073741824 [ 3.247492] virtio_blk virtio3: [vdc] 5120000 512-byte logical blocks (2.62 GB/2.44 GiB) [ 3.249194] vdc: detected capacity change from 0 to 2621440000 [ 3.261898] virtio_blk virtio4: [vdd] 5120000 512-byte logical blocks (2.62 GB/2.44 GiB) [ 3.263569] vdd: detected capacity change from 0 to 2621440000 [ 3.293317] virtio_blk virtio5: [vde] 8388608 512-byte logical blocks (4.29 GB/4.00 GiB) [ 3.296508] vde: detected capacity change from 0 to 4294967296 [ 3.311862] virtio_blk virtio6: [vdf] 8388608 512-byte logical blocks (4.29 GB/4.00 GiB) [ 3.314961] vdf: detected capacity change from 0 to 4294967296 [ 3.321364] libphy: Fixed MDIO Bus: probed [ 3.327904] usbcore: registered new interface driver usbserial_generic [ 3.329695] usbserial: USB Serial support registered for generic [ 3.331636] i8042: PNP: PS/2 Controller [PNP0303:KBD,PNP0f13:MOU] at 0x60,0x64 irq 1,12 [ 3.335176] serio: i8042 KBD port at 0x60,0x64 irq 1 [ 3.336914] serio: i8042 AUX port at 0x60,0x64 irq 12 [ 3.339836] mousedev: PS/2 mouse device common for all mice [ 3.342274] rtc_cmos 00:05: RTC can wake from S4 [ 3.344415] input: AT Translated Set 2 keyboard as /devices/platform/i8042/serio0/input/input1 [ 3.345400] rtc_cmos 00:05: registered as rtc0 [ 3.348924] rtc_cmos 00:05: alarms up to one day, y3k, 242 bytes nvram, hpet irqs [ 3.351639] intel_pstate: CPU model not supported [ 3.354326] input: VirtualPS/2 VMware VMMouse as /devices/platform/i8042/serio1/input/input4 [ 3.356916] hid: raw HID events driver (C) Jiri Kosina [ 3.360062] usbcore: registered new interface driver usbhid [ 3.360067] input: VirtualPS/2 VMware VMMouse as /devices/platform/i8042/serio1/input/input3 [ 3.361353] usbhid: USB HID core driver [ 3.364839] drop_monitor: Initializing network drop monitor service [ 3.367125] Initializing XFRM netlink socket [ 3.369160] NET: Registered protocol family 10 [ 3.372173] Segment Routing with IPv6 [ 3.373127] NET: Registered protocol family 17 [ 3.374488] mpls_gso: MPLS GSO support [ 3.379296] RAS: Correctable Errors collector initialized. [ 3.381406] AVX version of gcm_enc/dec engaged. [ 3.383098] AES CTR mode by8 optimization enabled [ 3.450066] sched_clock: Marking stable (3450028262, 0)->(4379984700, -929956438) [ 3.452251] registered taskstats version 1 [ 3.453861] Loading compiled-in X.509 certificates [ 3.455592] zswap: loaded using pool lzo/zbud [ 3.472601] Key type big_key registered [ 3.482642] Key type encrypted registered [ 3.484180] ima: No TPM chip found, activating TPM-bypass! [ 3.486186] ima: Allocated hash algorithm: sha1 [ 3.488037] ima: No architecture policies found [ 3.490078] evm: Initialising EVM extended attributes: [ 3.492189] evm: security.selinux [ 3.493582] evm: security.ima [ 3.494814] evm: security.capability [ 3.496254] evm: HMAC attrs: 0x1 [ 3.498788] rtc_cmos 00:05: setting system clock to 2026-08-27 01:10:10 UTC (1787793010) [ 3.505204] debug: unmapping init [mem 0xffffffffa9a03000-0xffffffffa9bfffff] [ 3.508502] debug: unmapping init [mem 0xffffffffa8782000-0xffffffffa8a58fff] [ 3.518093] Write protecting the kernel read-only data: 28672k [ 3.520745] debug: unmapping init [mem 0xffffffffa6e03000-0xffffffffa6ffffff] [ 3.523615] debug: unmapping init [mem 0xffffffffa7714000-0xffffffffa77fffff] [ 3.557612] systemd[1]: systemd 239 (239-82.el8_10.5) running in system mode. (+PAM +AUDIT +SELINUX +IMA -APPARMOR +SMACK +SYSVINIT +UTMP +LIBCRYPTSETUP +GCRYPT +GNUTLS +ACL +XZ +LZ4 +SECCOMP +BLKID +ELFUTILS +KMOD +IDN2 -IDN +PCRE2 default-hierarchy=legacy) [ 3.566632] systemd[1]: Detected virtualization kvm. [ 3.568594] systemd[1]: Detected architecture x86-64. [ 3.570418] systemd[1]: Running in initial RAM disk. Welcome to Rocky Linux 8.10 (Green Obsidian) dracut-049-233.git20240115.el8 (Initramfs)! [ 3.603811] systemd[1]: No hostname configured. [ 3.607584] systemd[1]: Set hostname to . [ 3.609939] random: systemd: uninitialized urandom read (16 bytes read) [ 3.612640] systemd[1]: Initializing machine ID from random generator. [ 3.664351] random: ln: uninitialized urandom read (6 bytes read) [ 3.746585] random: systemd: uninitialized urandom read (16 bytes read) [ 3.749568] systemd[1]: Reached target Local File Systems. [ OK ] Reached target Local File Systems. [ 3.754706] systemd[1]: Listening on udev Kernel Socket. [ OK ] Listening on udev Kernel Socket. [ 3.759245] systemd[1]: Reached target Timers. [ OK ] Reached target Timers. [ OK ] Listening on Journal Socket (/dev/log). [ OK ] Reached target Initrd Root Device. [ OK ] Listening on Journal Socket. Starting Journal Service... Starting Create list of required st…ce nodes for the current kernel... Starting Setup Virtual Console... [ OK ] Started Dispatch Password Requests to Console Directory Watch. [ OK ] Reached target Paths. Starting Apply Kernel Variables... [ OK ] Listening on udev Control Socket. [ OK ] Reached target Sockets. [ OK ] Started Memstrack Anylazing Service. [ OK ] Reached target Swap. [ OK ] Reached target Slices. [ OK ] Reached target Local Encrypted Volumes. Starting Create Volatile Files and Directories... [ OK ] Started Create list of required sta…vice nodes for the current kernel. [ OK ] Started Setup Virtual Console. [ OK ] Started Apply Kernel Variables. Starting dracut cmdline hook... Starting Create Static Device Nodes in /dev... [ OK ] Started Journal Service. [ OK ] Started Create Volatile Files and Directories. [ OK ] Started Create Static Device Nodes in /dev. [ OK ] Started dracut cmdline hook. Starting dracut pre-udev hook... [ 4.352386] device-mapper: uevent: version 1.0.3 [ 4.354779] device-mapper: ioctl: 4.46.0-ioctl (2022-02-22) initialised: dm-devel@redhat.com [ OK ] Started dracut pre-udev hook. Starting udev Kernel Device Manager... [ OK ] Started udev Kernel Device Manager. Starting dracut pre-trigger hook... [ OK ] Started dracut pre-trigger hook. Starting udev Coldplug all Devices... Mounting Kernel Configuration File System... [ OK ] Mounted Kernel Configuration File System. [ OK ] Started udev Coldplug all Devices. Starting dracut initqueue hook... [ OK ] Reached target System Initialization. [ OK ] Reached target Basic System. [ OK ] Started Hardware RNG Entropy Gatherer Daemon. [ 5.132470] virtio_net virtio0 ens2: renamed from eth0 [ 5.142397] random: fast init done [ 5.266189] scsi host0: ata_piix [ 5.281935] scsi host1: ata_piix [ 5.287245] ata1: PATA max MWDMA2 cmd 0x1f0 ctl 0x3f6 bmdma 0xc320 irq 14 [ 5.289930] ata2: PATA max MWDMA2 cmd 0x170 ctl 0x376 bmdma 0xc328 irq 15 [ 9.024712] dracut-initqueue[581]: RTNETLINK answers: File exists [ 10.002411] random: crng init done [ 10.003891] random: 7 urandom warning(s) missed due to ratelimiting Starting nbd nbd0... [ OK ] Started nbd nbd0. [ OK ] Started dracut initqueue hook. Mounting /sysroot... [ OK ] Reached target Remote File Systems (Pre). [ OK ] Reached target Remote File Systems. [ 10.443283] EXT4-fs (nbd0): mounted filesystem with ordered data mode. Opts: (null) [ OK ] Mounted /sysroot. [ OK ] Reached target Initrd Root File System. Starting Reload Configuration from the Real Root... [ OK ] Started Reload Configuration from the Real Root. [ OK ] Reached target Initrd File Systems. [ OK ] Reached target Initrd Default Target. Starting dracut pre-pivot and cleanup hook... [ OK ] Started dracut pre-pivot and cleanup hook. Starting Cleaning Up and Shutting Down Daemons... Stopping Hardware RNG Entropy Gatherer Daemon... [ OK ] Stopped target Timers. [ OK ] Stopped dracut pre-pivot and cleanup hook. [ OK ] Stopped target Initrd Default Target. [ OK ] Stopped target Remote File Systems. [ OK ] Stopped target Remote File Systems (Pre). [ OK ] Stopped target Initrd Root Device. [ OK ] Stopped dracut initqueue hook. [ OK ] Stopped Hardware RNG Entropy Gatherer Daemon. [ OK ] Stopped target Basic System. [ OK ] Stopped target Slices. [ OK ] Stopped target Paths. [ OK ] Stopped target Sockets. [ OK ] Stopped target System Initialization. [ OK ] Stopped Create Volatile Files and Directories. [ OK ] Stopped target Local File Systems. [ OK ] Stopped udev Coldplug all Devices. [ OK ] Stopped dracut pre-trigger hook. Stopping udev Kernel Device Manager... [ OK ] Stopped target Local Encrypted Volumes. [ OK ] Stopped Dispatch Password Requests to Console Directory Watch. [ OK ] Stopped target Swap. [ OK ] Stopped Apply Kernel Variables. [ OK ] Started Cleaning Up and Shutting Down Daemons. [ OK ] Stopped udev Kernel Device Manager. [ OK ] Stopped dracut pre-udev hook. [ OK ] Stopped dracut cmdline hook. [ OK ] Stopped Create Static Device Nodes in /dev. [ OK ] Stopped Create list of required sta…vice nodes for the current kernel. [ OK ] Closed udev Control Socket. [ OK ] Closed udev Kernel Socket. Starting Cleanup udevd DB... [ OK ] Started Cleanup udevd DB. [ OK ] Reached target Switch Root. Starting Switch Root... [ 11.489794] printk: systemd: 23 output lines suppressed due to ratelimiting [ 11.725819] SELinux: Disabled at runtime. [ 11.783985] systemd[1]: systemd 239 (239-82.el8_10.5) running in system mode. (+PAM +AUDIT +SELINUX +IMA -APPARMOR +SMACK +SYSVINIT +UTMP +LIBCRYPTSETUP +GCRYPT +GNUTLS +ACL +XZ +LZ4 +SECCOMP +BLKID +ELFUTILS +KMOD +IDN2 -IDN +PCRE2 default-hierarchy=legacy) [ 11.792812] systemd[1]: Detected virtualization kvm. [ 11.795024] systemd[1]: Detected architecture x86-64. Welcome to Rocky Linux 8.10 (Green Obsidian)! [ 12.266238] systemd[1]: initrd-switch-root.service: Succeeded. [ 12.268756] systemd[1]: Stopped Switch Root. [ OK ] Stopped Switch Root. [ 12.272977] systemd[1]: systemd-journald.service: Service has no hold-off time (RestartSec=0), scheduling restart. [ 12.275335] systemd[1]: systemd-journald.service: Scheduled restart job, restart counter is at 1. [ 12.277897] systemd[1]: Stopped Journal Service. [ OK ] Stopped Journal Service. [ 12.287338] systemd[1]: Starting Journal Service... Starting Journal Service... [ 12.292030] systemd[1]: Created slice system-serial\x2dgetty.slice. [ OK ] Created slice system-serial\x2dgetty.slice. Mounting POSIX Message Queue File System... Mounting Kernel Debug File System... Starting Create list of required st…ce nodes for the current kernel... [ OK ] Created slice system-getty.slice. [ OK ] Listening on initctl Compatibility Named Pipe. Mounting Huge Pages File System... [ OK ] Listening on Process Core Dump Socket. [ OK ] Reached target rpc_pipefs.target. Starting Apply Kernel Variables... Starting Remount Root and Kernel File Systems... [ OK ] Started Dispatch Password Requests to Console Directory Watch. [FAILED] Failed to set up automount Arbitrar…rmats File System Automount Point. See 'systemctl status proc-sys-fs-binfmt_misc.automount' for details. [ OK ] Listening on udev Control Socket. [ OK ] Listening on udev Kernel Socket. [ OK ] Listening on RPCbind Server Activation Socket. [ OK ] Stopped target Switch Root. [ OK ] Stopped target Initrd File Systems. [ OK ] Stopped target Initrd Root File System. [ OK ] Created slice User and Session Slice. [ OK ] Reached target Slices. [ OK ] Reached target RPC Port Mapper. [ OK ] Started Forward Password Requests to Wall Directory Watch. [ OK ] Reached target Local Encrypted Volumes. [ OK ] Reached target Paths. Activating swap /dev/disk/by-label/SWAP... [ OK ] Created slice system-sshd\x2dkeygen.slice. Starting udev Coldplug all Devices... [ 12.484654] Adding 1048572k swap on /dev/vdb. Priority:-2 extents:1 across:1048572k FS [ OK ] Started Journal Service. [ OK ] Mounted POSIX Message Queue File System. [ OK ] Mounted Kernel Debug File System. [ OK ] Started Create list of required sta…vice nodes for the current kernel. [ OK ] Mounted Huge Pages File System. [ OK ] Started Apply Kernel Variables. [FAILED] Failed to start Remount Root and Kernel File Systems. See 'systemctl status systemd-remount-fs.service' for details. [ OK ] Activated swap /dev/disk/by-label/SWAP. [ OK ] Reached target Swap. Starting Configure read-only root support... Starting Create Static Device Nodes in /dev... Starting Flush Journal to Persistent Storage... [ OK ] Started Flush Journal to Persistent Storage. [ OK ] Started Create Static Device Nodes in /dev. Starting udev Kernel Device Manager... [ OK ] Reached target Local File Systems (Pre). Mounting /mnt... Mounting /home/green/git/lustre-release... [ OK ] Mounted /mnt. [ OK ] Started udev Coldplug all Devices. [ 12.826215] squashfs: version 4.0 (2009/01/31) Phillip Lougher [ OK ] Mounted /home/green/git/lustre-release. [ OK ] Started udev Kernel Device Manager. [ 13.124725] piix4_smbus 0000:00:01.3: SMBus Host Controller at 0x700, revision 0 [ 13.172462] input: PC Speaker as /devices/platform/pcspkr/input/input5 [ 13.333437] RAPL PMU: API unit is 2^-32 Joules, 0 fixed counters, 10737418240 ms ovfl timer [ 13.348088] EDAC sbridge: Ver: 1.1.2 [ 15.047057] Key type dns_resolver registered [ 15.369885] NFS: Registering the id_resolver key type [ 15.371896] Key type id_resolver registered [ 15.373242] Key type id_legacy registered [ OK ] Started Configure read-only root support. Starting Load/Save Random Seed... [ OK ] Reached target Local File Systems. Starting Mark the need to relabel after reboot... Starting Rebuild Dynamic Linker Cache... Starting Create Volatile Files and Directories... [ OK ] Started Load/Save Random Seed. [ OK ] Started Mark the need to relabel after reboot. [ OK ] Started Create Volatile Files and Directories. Starting RPC Bind... Starting Update UTMP about System Boot/Shutdown... [ OK ] Started RPC Bind. [ OK ] Started Update UTMP about System Boot/Shutdown. [ OK ] Started Rebuild Dynamic Linker Cache. Starting Update is Completed... [ OK ] Started Update is Completed. [ OK ] Reached target System Initialization. [ OK ] Started Daily Cleanup of Temporary Directories. [ OK ] Listening on D-Bus System Message Bus Socket. [ OK ] Reached target Sockets. [ OK ] Started daily update of the root trust anchor for DNSSEC. [ OK ] Reached target Basic System. Starting Restore /run/initramfs on shutdown... [ OK ] Started irqbalance daemon. [ OK ] Started D-Bus System Message Bus. Starting Network Manager... [ OK ] Reached target sshd-keygen.target. Starting Login Service... [ OK ] Started dnf makecache --timer. [ OK ] Reached target Timers. [ OK ] Started Hardware RNG Entropy Gatherer Daemon. [ OK ] Started Restore /run/initramfs on shutdown. [ OK ] Started Login Service. [ OK ] Started Network Manager. [ OK ] Reached target Network. Starting GSSAPI Proxy Daemon... Starting Dynamic System Tuning Daemon... Starting OpenSSH server daemon... Starting Network Manager Wait Online... [ OK ] Started GSSAPI Proxy Daemon. [ OK ] Reached target NFS client services. [ OK ] Reached target Remote File Systems (Pre). [ OK ] Reached target Remote File Systems. Starting Permit User Sessions... Starting Hostname Service... [ OK ] Started OpenSSH server daemon. [ OK ] Started Permit User Sessions. [ OK ] Started Command Scheduler. [ OK ] Started Serial Getty on ttyS1. [ OK ] Started Getty on tty1. [ OK ] Started Serial Getty on ttyS0. [ OK ] Reached target Login Prompts. [ OK ] Started Hostname Service. Starting Network Manager Script Dispatcher Service... [ OK ] Started Network Manager Script Dispatcher Service. [ OK ] Started Network Manager Wait Online. [ OK ] Reached target Network is Online. Starting Crash recovery kernel arming... Starting System Logging Service... Starting Notify NFS peers of a restart... [ OK ] Started Notify NFS peers of a restart. [ OK ] Started System Logging Service. Starting Authorization Manager... [ OK ] Started Authorization Manager. [ OK ] Started Dynamic System Tuning Daemon. [ OK ] Reached target Multi-User System. [ OK ] Reached target Graphical Interface. Starting Update UTMP about System Runlevel Changes... [ OK ] Started Update UTMP about System Runlevel Changes. Rocky Linux 8.10 (Green Obsidian) Kernel 4.18.0rh8.10-debug on an x86_64 oleg158-server login: [ 84.534919] libcfs: loading out-of-tree module taints kernel. [ 84.641580] Key type ._llcrypt registered [ 84.647270] Key type .llcrypt registered [ 84.805379] Lustre: DEBUG MARKER: oleg158-server.virtnet: executing set_hostid [ 104.660150] Lustre: DEBUG MARKER: oleg158-server.virtnet: executing load_modules_local [ 106.549332] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 106.556580] alg: No test for adler32 (adler32-zlib) [ 108.028939] Lustre: Lustre: Build Version: 2.17.57_48_g018c4cb [ 109.085084] LNet: Added LNI 192.168.201.158@tcp [8/256/0/180] [ 111.087669] Key type lgssc registered [ 113.691864] hrtimer: interrupt took 7659258 ns [ 113.786744] Lustre: Echo OBD driver; http://www.lustre.org/ [ 133.092038] ZFS: Loaded module v2.3.2-1, ZFS pool version 5000, ZFS filesystem version 5 [ 178.773979] Lustre: DEBUG MARKER: oleg158-server.virtnet: executing load_modules_local [ 192.243252] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 192.279352] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 193.552583] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 193.579313] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 193.659677] Lustre: lustre-MDT0000: new disk, initializing [ 193.756664] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 193.777292] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 198.963964] Lustre: DEBUG MARKER: oleg158-server.virtnet: executing set_default_debug vfstrace rpctrace dlmtrace neterror ha config ioctl super lfsck all [ 211.333500] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 211.430634] Lustre: 6508:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 211.476709] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [ 211.482970] Lustre: Skipped 1 previous similar message [ 211.606339] Lustre: lustre-MDT0001: new disk, initializing [ 211.725067] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 211.763209] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [ 211.781942] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 215.872452] Lustre: DEBUG MARKER: oleg158-server.virtnet: executing set_default_debug vfstrace rpctrace dlmtrace neterror ha config ioctl super lfsck all [ 220.465537] Lustre: Modifying parameter general.debug_raw_pointers=Y in log params [ 230.341625] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 230.493562] Lustre: lustre-OST0000: new disk, initializing [ 230.495607] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 230.498973] Lustre: 8446:0:(osd_compat.c:1353:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 230.558062] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 232.232692] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [ 232.243353] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 232.348518] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 236.800165] Lustre: DEBUG MARKER: oleg158-server.virtnet: executing set_default_debug vfstrace rpctrace dlmtrace neterror ha config ioctl super lfsck all [ 250.463253] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 250.659981] Lustre: lustre-OST0001: new disk, initializing [ 250.666267] Lustre: srv-lustre-OST0001: No data found on store. Initialize space. [ 250.674739] Lustre: 9518:0:(osd_compat.c:1353:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 250.739813] Lustre: lustre-OST0001: Imperative Recovery not enabled, recovery window 60-180 [ 256.883187] Lustre: DEBUG MARKER: oleg158-server.virtnet: executing set_default_debug vfstrace rpctrace dlmtrace neterror ha config ioctl super lfsck all [ 260.654567] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x00000002c0000400-0x0000000300000400]:1:ost [ 260.670087] Lustre: cli-lustre-OST0001-super: Allocated super-sequence [0x00000002c0000400-0x0000000300000400]:1:ost] [ 260.757052] Lustre: lustre-OST0001-osc-MDT0000: update sequence from 0x100010000 to 0x2c0000401 [ 269.292194] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 277.014885] Lustre: Setting parameter general.lod.*.mdt_hash=crush in log params [ 283.186052] Lustre: DEBUG MARKER: oleg158-server.virtnet: executing check_logdir /tmp/testlogs/ [ 288.442702] Lustre: DEBUG MARKER: oleg158-server.virtnet: executing yml_node [ 293.829049] Lustre: DEBUG MARKER: Client: 2.17.57.48 [ 296.473854] Lustre: DEBUG MARKER: MDS: 2.17.57.48 [ 299.576030] Lustre: DEBUG MARKER: OSS: 2.17.57.48 [ 301.671149] Lustre: DEBUG MARKER: -----============= acceptance-small: conf-sanity ============----- Wed Aug 26 21:15:07 EDT 2026 [ 319.239351] Lustre: DEBUG MARKER: excepting tests: 102 106 115 32newtarball [ 321.025727] Lustre: DEBUG MARKER: skipping tests SLOW=no: 45 69 106 111 114 [ 339.937688] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 339.965241] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 339.980109] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 342.500974] Lustre: lustre-MDT0000-lwp-OST0000: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 342.505964] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 342.521490] Lustre: Skipped 2 previous similar messages [ 342.539153] Lustre: Skipped 3 previous similar messages [ 344.743204] Lustre: server umount lustre-MDT0000 complete [ 348.870853] LustreError: 7465:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1787793355 with bad export cookie 4837577319687934851 [ 348.873905] LustreError: MGC192.168.201.158@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 348.883740] LustreError: 7465:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 1 previous similar message [ 352.735716] LustreError: 6516:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 352.736280] Lustre: lustre-MDT0001-lwp-OST0001: Connection to lustre-MDT0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 352.745027] Lustre: lustre-MDT0001: Not available for connect from 0@lo (stopping) [ 352.746141] LustreError: 6516:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 6 previous similar messages [ 352.780501] Lustre: Skipped 1 previous similar message [ 353.769145] LustreError: 6517:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 353.794682] LustreError: 6517:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 1 previous similar message [ 355.380223] Lustre: server umount lustre-MDT0001 complete [ 359.392818] Lustre: server umount lustre-OST0000 complete [ 363.109861] Lustre: server umount lustre-OST0001 complete [ 371.687979] Lustre: DEBUG MARKER: oleg158-server.virtnet: executing set_hostid [ 381.461911] Lustre: DEBUG MARKER: oleg158-server.virtnet: executing load_modules_local [ 424.071350] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 424.424357] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 424.466400] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 424.566218] Lustre: lustre-MDT0000: new disk, initializing [ 424.660719] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 424.681329] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 430.646689] Lustre: DEBUG MARKER: oleg158-server.virtnet: executing set_default_debug -1 all [ 444.233508] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 444.408875] Lustre: 15074:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 444.515872] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [ 444.528968] Lustre: Skipped 1 previous similar message [ 444.646106] Lustre: lustre-MDT0001: new disk, initializing [ 444.792380] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 444.871236] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [ 444.895499] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 450.573217] Lustre: DEBUG MARKER: oleg158-server.virtnet: executing set_default_debug -1 all [ 461.174565] Lustre: DEBUG MARKER: oleg158-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 467.578561] Lustre: DEBUG MARKER: oleg158-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 475.264312] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 475.545545] Lustre: lustre-OST0000: new disk, initializing [ 475.550959] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 475.560692] Lustre: 17043:0:(osd_compat.c:1353:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 475.700886] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 476.884046] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [ 476.904637] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 477.131690] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 482.638599] Lustre: DEBUG MARKER: oleg158-server.virtnet: executing set_default_debug -1 all [ 493.797524] Lustre: DEBUG MARKER: oleg158-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 499.338781] Lustre: DEBUG MARKER: oleg158-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 [ 499.503699] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [ 504.239608] Lustre: DEBUG MARKER: oleg158-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid 50 [ 504.494696] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [ 507.878255] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 507.881719] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 507.887086] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 507.911330] Lustre: Skipped 1 previous similar message [ 507.925591] Lustre: Skipped 4 previous similar messages [ 513.058240] Lustre: server umount lustre-OST0000 complete [ 518.112987] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 518.118205] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 518.130441] Lustre: Skipped 1 previous similar message [ 518.153123] Lustre: Skipped 1 previous similar message [ 523.782496] Lustre: server umount lustre-MDT0000 complete [ 527.344516] LustreError: 15086:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 527.365609] LustreError: 15086:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 1 previous similar message [ 528.979037] LustreError: 15066:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1787793535 with bad export cookie 4837577319687936384 [ 528.984330] LustreError: MGC192.168.201.158@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 528.984880] LustreError: 15066:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 2 previous similar messages [ 529.441842] Lustre: server umount lustre-MDT0001 complete [ 535.876885] Lustre: DEBUG MARKER: == conf-sanity test 76a: set permanent params with lctl across mounts ========================================================== 21:19:01 (1787793541) [ 543.720580] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 544.150961] LustreError: 19311:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 544.269216] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 549.183888] Lustre: DEBUG MARKER: oleg158-server.virtnet: executing set_default_debug -1 all [ 549.348579] LustreError: 19312:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 559.186583] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 564.336630] Lustre: DEBUG MARKER: oleg158-server.virtnet: executing set_default_debug -1 all [ 572.769522] Lustre: DEBUG MARKER: oleg158-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 580.661713] Lustre: DEBUG MARKER: oleg158-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 589.240968] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 596.688807] Lustre: DEBUG MARKER: oleg158-server.virtnet: executing set_default_debug -1 all [ 604.800858] Lustre: DEBUG MARKER: oleg158-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 608.900925] Lustre: Modifying parameter general.osc.*.max_dirty_mb=452 in log params [ 615.907403] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 615.917649] LustreError: Skipped 1 previous similar message [ 615.923290] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 615.942035] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 615.953030] Lustre: Skipped 2 previous similar messages [ 622.055802] Lustre: server umount lustre-MDT0000 complete [ 625.601244] LustreError: 19291:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1787793632 with bad export cookie 4837577319687937266 [ 625.605697] LustreError: MGC192.168.201.158@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 625.610978] LustreError: 19291:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 1 previous similar message [ 625.635061] LustreError: 20492:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 625.652027] LustreError: 20492:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 1 previous similar message [ 625.988550] Lustre: server umount lustre-MDT0001 complete [ 631.777372] Lustre: server umount lustre-OST0000 complete [ 655.990761] Lustre: DEBUG MARKER: oleg158-server.virtnet: executing load_modules_local [ 667.268489] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 667.784955] LustreError: 23691:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 667.912382] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 667.920256] Lustre: Skipped 2 previous similar messages [ 673.065582] Lustre: DEBUG MARKER: oleg158-server.virtnet: executing set_default_debug -1 all [ 681.850832] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 687.242435] Lustre: DEBUG MARKER: oleg158-server.virtnet: executing set_default_debug -1 all [ 690.614212] Lustre: Setting parameter general.debug_raw_pointers=Y in log params [ 698.097964] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 706.134667] Lustre: DEBUG MARKER: oleg158-server.virtnet: executing set_default_debug -1 all [ 716.329758] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 716.628971] Lustre: lustre-OST0001: new disk, initializing [ 716.636890] Lustre: srv-lustre-OST0001: No data found on store. Initialize space. [ 716.646689] Lustre: 26006:0:(osd_compat.c:1353:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 718.093160] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x00000002c0000400-0x0000000300000400]:1:ost [ 718.105710] Lustre: cli-lustre-OST0001-super: Allocated super-sequence [0x00000002c0000400-0x0000000300000400]:1:ost] [ 718.230318] Lustre: lustre-OST0001-osc-MDT0000: update sequence from 0x100010000 to 0x2c0000401 [ 725.809512] Lustre: DEBUG MARKER: oleg158-server.virtnet: executing set_default_debug -1 all [ 737.900520] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 743.247570] Lustre: Setting parameter general.lod.*.mdt_hash=crush in log params [ 759.270723] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 759.275808] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 759.277240] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 759.277246] Lustre: Skipped 4 previous similar messages [ 759.303182] Lustre: Skipped 6 previous similar messages [ 763.339851] Lustre: server umount lustre-MDT0000 complete [ 764.398201] LustreError: 23687:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 764.426508] LustreError: 23687:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 5 previous similar messages [ 767.026919] LustreError: 23673:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1787793774 with bad export cookie 4837577319687938113 [ 767.028118] LustreError: MGC192.168.201.158@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 767.045938] LustreError: 23673:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 1 previous similar message [ 767.482695] Lustre: server umount lustre-MDT0001 complete [ 781.824251] Lustre: server umount lustre-OST0000 complete [ 796.021081] Lustre: server umount lustre-OST0001 complete [ 812.813228] Lustre: DEBUG MARKER: oleg158-server.virtnet: executing load_modules_local [ 822.873125] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 823.360927] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 823.374667] Lustre: Skipped 3 previous similar messages [ 828.200625] Lustre: DEBUG MARKER: oleg158-server.virtnet: executing set_default_debug -1 all [ 833.507392] LustreError: 29030:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 833.535467] LustreError: 29030:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 2 previous similar messages [ 836.071932] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 840.745993] Lustre: DEBUG MARKER: oleg158-server.virtnet: executing set_default_debug -1 all [ 843.890310] Lustre: 30176:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify general/debug_raw_pointers=Y (mode = 0) failed: rc = -17 [ 851.928101] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 859.634108] Lustre: DEBUG MARKER: oleg158-server.virtnet: executing set_default_debug -1 all [ 870.081339] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 876.812380] Lustre: DEBUG MARKER: oleg158-server.virtnet: executing set_default_debug -1 all [ 885.581195] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 889.587981] Lustre: 32056:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify general/lod.*.mdt_hash=crush (mode = 0) failed: rc = -17 [ 895.976696] Lustre: lustre-MDT0000-lwp-OST0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 895.983767] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 895.997332] Lustre: Skipped 1 previous similar message [ 896.015724] Lustre: Skipped 6 previous similar messages [ 902.298817] Lustre: server umount lustre-MDT0000 complete [ 905.874056] LustreError: 29012:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1787793912 with bad export cookie 4837577319687939408 [ 905.883487] LustreError: MGC192.168.201.158@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 906.155450] Lustre: server umount lustre-MDT0001 complete [ 920.259640] Lustre: server umount lustre-OST0000 complete [ 922.400142] Lustre: 3645:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1787793913/real 1787793913] req@ffffa039037fa680 x1874636954702208/t0(0) o400->lustre-MDT0001-lwp-OST0001@0@lo:12/10 lens 224/224 e 0 to 1 dl 1787793929 ref 1 fl Rpc:XNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 [ 924.740269] Lustre: server umount lustre-OST0001 complete [ 934.913126] Lustre: DEBUG MARKER: == conf-sanity test 76b: verify params log setup correctly ========================================================== 21:25:40 (1787793940) [ 965.055758] Lustre: DEBUG MARKER: oleg158-server.virtnet: executing load_modules_local [ 975.069933] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 975.350686] LustreError: 34861:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 975.367551] LustreError: 34861:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 4 previous similar messages [ 979.509149] Lustre: DEBUG MARKER: oleg158-server.virtnet: executing set_default_debug -1 all [ 988.881144] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 993.960439] Lustre: DEBUG MARKER: oleg158-server.virtnet: executing set_default_debug -1 all [ 997.495878] Lustre: 36010:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify general/debug_raw_pointers=Y (mode = 0) failed: rc = -17 [ 1005.122498] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 1011.344405] Lustre: DEBUG MARKER: oleg158-server.virtnet: executing set_default_debug -1 all [ 1019.647169] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 1026.098781] Lustre: DEBUG MARKER: oleg158-server.virtnet: executing set_default_debug -1 all [ 1034.095569] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 1038.163335] Lustre: 37890:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify general/lod.*.mdt_hash=crush (mode = 0) failed: rc = -17 [ 1045.475952] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 1045.491964] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 1045.509262] Lustre: Skipped 3 previous similar messages [ 1045.514054] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 1045.522927] Lustre: Skipped 4 previous similar messages [ 1051.351938] Lustre: server umount lustre-MDT0000 complete [ 1055.392310] LustreError: 34843:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1787794062 with bad export cookie 4837577319687940514 [ 1055.408193] LustreError: 34843:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 1055.422212] LustreError: MGC192.168.201.158@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 1055.922264] Lustre: server umount lustre-MDT0001 complete [ 1069.298668] Lustre: server umount lustre-OST0000 complete [ 1083.827068] Lustre: server umount lustre-OST0001 complete [ 1093.132799] Lustre: DEBUG MARKER: == conf-sanity test 76c: verify changelog_mask is applied with lctl set_param -P ========================================================== 21:28:18 (1787794098) [ 1110.817753] Lustre: DEBUG MARKER: oleg158-server.virtnet: executing load_modules_local [ 1121.757590] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1122.391611] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 1122.401353] Lustre: Skipped 7 previous similar messages [ 1127.901579] Lustre: DEBUG MARKER: oleg158-server.virtnet: executing set_default_debug -1 all [ 1138.956386] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1145.317542] Lustre: DEBUG MARKER: oleg158-server.virtnet: executing set_default_debug -1 all [ 1148.959350] Lustre: 41181:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify general/debug_raw_pointers=Y (mode = 0) failed: rc = -17 [ 1156.273075] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 1164.193598] Lustre: DEBUG MARKER: oleg158-server.virtnet: executing set_default_debug -1 all [ 1172.941622] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 1179.364767] Lustre: DEBUG MARKER: oleg158-server.virtnet: executing set_default_debug -1 all [ 1187.812240] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 1191.778473] Lustre: 43059:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify general/lod.*.mdt_hash=crush (mode = 0) failed: rc = -17 [ 1192.920231] Lustre: Setting parameter general.mdd.*.changelog_mask=-CLOSE in log params [ 1192.928643] Lustre: Skipped 1 previous similar message [ 1197.371433] Lustre: lustre-MDT0000: Not available for connect from 192.168.201.58@tcp (stopping) [ 1197.387206] Lustre: Skipped 4 previous similar messages [ 1199.075586] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 1199.097572] Lustre: Skipped 5 previous similar messages [ 1203.541603] Lustre: server umount lustre-MDT0000 complete [ 1207.795026] LustreError: 40011:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1787794214 with bad export cookie 4837577319687941620 [ 1207.800891] LustreError: MGC192.168.201.158@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 1207.809179] LustreError: 40011:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 1214.331865] Lustre: server umount lustre-MDT0001 complete [ 1223.279213] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1232.864924] LustreError: 3642:0:(client.c:1404:ptlrpc_import_delay_req()) @@@ invalidate in flight req@ffffa03a395cdf80 x1874636954804096/t0(0) o250->MGC192.168.201.158@tcp@0@lo:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl Rpc:NQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 [ 1233.250663] LustreError: 41544:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 1233.276103] LustreError: 41544:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 25 previous similar messages [ 1237.643597] Lustre: DEBUG MARKER: oleg158-server.virtnet: executing set_default_debug -1 all [ 1243.626284] Lustre: lustre-MDT0000-lwp-OST0000: Connection restored to 0@lo (at 0@lo) [ 1246.129848] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1250.887473] Lustre: DEBUG MARKER: oleg158-server.virtnet: executing set_default_debug -1 all [ 1251.828791] Lustre: lustre-MDT0001-lwp-OST0000: Connection restored to 0@lo (at 0@lo) [ 1251.847376] Lustre: Skipped 1 previous similar message [ 1258.824504] Lustre: DEBUG MARKER: oleg158-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 1260.454678] Lustre: DEBUG MARKER: mdc.lustre-MDT0000-mdc-*.mds_server_uuid in FULL state after 0 sec [ 1267.684334] Lustre: DEBUG MARKER: oleg158-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 1270.051912] Lustre: DEBUG MARKER: mdc.lustre-MDT0001-mdc-*.mds_server_uuid in FULL state after 0 sec [ 1277.418517] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 1283.912234] Lustre: server umount lustre-MDT0000 complete [ 1289.203690] LustreError: 40011:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1787794296 with bad export cookie 4837577319687942747 [ 1289.207085] LustreError: MGC192.168.201.158@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 1289.223207] LustreError: 40011:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 3 previous similar messages [ 1290.066369] Lustre: server umount lustre-MDT0001 complete [ 1304.566719] Lustre: server umount lustre-OST0000 complete [ 1317.396896] Lustre: server umount lustre-OST0001 complete [ 1327.493532] Lustre: DEBUG MARKER: == conf-sanity test 76d: verify llite.*.xattr_cache can be set by 'lctl set_param -P' correctly ========================================================== 21:32:12 (1787794332) [ 1346.440836] Lustre: DEBUG MARKER: oleg158-server.virtnet: executing load_modules_local [ 1359.216134] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1364.448995] Lustre: DEBUG MARKER: oleg158-server.virtnet: executing set_default_debug -1 all [ 1373.858461] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1379.228743] Lustre: DEBUG MARKER: oleg158-server.virtnet: executing set_default_debug -1 all [ 1382.412544] Lustre: 48826:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify general/debug_raw_pointers=Y (mode = 0) failed: rc = -17 [ 1389.424533] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 1398.612557] Lustre: DEBUG MARKER: oleg158-server.virtnet: executing set_default_debug -1 all [ 1409.168314] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 1415.832820] Lustre: DEBUG MARKER: oleg158-server.virtnet: executing set_default_debug -1 all [ 1424.874451] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 1430.794460] Lustre: Setting parameter general.llite.*.xattr_cache=0 in log params [ 1445.611860] Lustre: server umount lustre-MDT0000 complete [ 1449.007354] LustreError: 47654:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1787794456 with bad export cookie 4837577319687943538 [ 1449.015122] LustreError: MGC192.168.201.158@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 1449.017925] LustreError: 47654:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 5 previous similar messages [ 1449.449686] Lustre: server umount lustre-MDT0001 complete [ 1464.478493] Lustre: server umount lustre-OST0000 complete [ 1466.335265] Lustre: 3645:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1787794457/real 1787794457] req@ffffa0390261b480 x1874636954896640/t0(0) o400->lustre-MDT0001-lwp-OST0001@0@lo:12/10 lens 224/224 e 0 to 1 dl 1787794473 ref 1 fl Rpc:XNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 [ 1466.378560] Lustre: lustre-MDT0001-lwp-OST0001: Connection to lustre-MDT0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 1466.387422] Lustre: Skipped 11 previous similar messages [ 1468.890448] Lustre: server umount lustre-OST0001 complete [ 1478.812869] Lustre: DEBUG MARKER: == conf-sanity test 77: comma-separated MGS NIDs and failover node NIDs ========================================================== 21:34:44 (1787794484) [ 1481.815249] Lustre: DEBUG MARKER: SKIP: conf-sanity test_77 mixed loopback and real device not working [ 1483.715754] Lustre: DEBUG MARKER: == conf-sanity test 78: run resize2fs on MDT and OST filesystems ========================================================== 21:34:49 (1787794489) [ 1495.046835] Lustre: DEBUG MARKER: oleg158-server.virtnet: executing set_hostid [ 1505.945914] Lustre: DEBUG MARKER: oleg158-server.virtnet: executing load_modules_local [ 1527.640831] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1527.967826] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 1528.062542] Lustre: lustre-MDT0000: new disk, initializing [ 1528.133630] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 1533.010728] Lustre: DEBUG MARKER: oleg158-server.virtnet: executing set_default_debug -1 all [ 1543.868855] Lustre: DEBUG MARKER: oleg158-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 1550.889099] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 1551.194712] Lustre: lustre-OST0000: new disk, initializing [ 1551.198886] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 1551.207269] Lustre: Skipped 1 previous similar message [ 1551.219571] Lustre: 54899:0:(osd_compat.c:1353:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 1552.729185] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:0:ost [ 1552.737226] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:0:ost] [ 1552.782848] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x240000400 [ 1557.689562] Lustre: DEBUG MARKER: oleg158-server.virtnet: executing set_default_debug -1 all [ 1567.350970] Lustre: DEBUG MARKER: oleg158-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 1569.575543] Lustre: DEBUG MARKER: create test files [ 1598.945774] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 1598.955908] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 1598.958834] Lustre: Skipped 22 previous similar messages [ 1604.826657] Lustre: server umount lustre-OST0000 complete [ 1608.721662] Lustre: server umount lustre-MDT0000 complete [ 1618.835382] Lustre: DEBUG MARKER: oleg158-server.virtnet: executing unload_modules_local [ 1620.874708] Key type lgssc unregistered [ 1621.153761] LNet: 56533:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 1621.158437] LNetError: 56533:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 1622.181784] LNet: Removed LNI 192.168.201.158@tcp [ 1623.115508] Key type .llcrypt unregistered [ 1623.120147] Key type ._llcrypt unregistered [ 1650.459222] Key type ._llcrypt registered [ 1650.461723] Key type .llcrypt registered [ 1650.558841] Lustre: DEBUG MARKER: oleg158-server.virtnet: executing load_modules_local [ 1651.223655] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 1651.342704] alg: No test for adler32 (adler32-zlib) [ 1652.343345] Lustre: Lustre: Build Version: 2.17.57_48_g018c4cb [ 1652.554803] LNet: Added LNI 192.168.201.158@tcp [8/256/0/180] [ 1654.215168] Key type lgssc registered [ 1655.385718] Lustre: Echo OBD driver; http://www.lustre.org/ [ 1661.574888] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 1661.596505] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1662.983540] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 1667.824326] Lustre: DEBUG MARKER: oleg158-server.virtnet: executing set_default_debug -1 all [ 1675.879461] Lustre: DEBUG MARKER: oleg158-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 1684.557127] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 1684.977642] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 1687.027340] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x240000400:99 to 0x240000400:129) [ 1687.122887] Lustre: 59261:0:(osd_internal.h:1460:osd_trans_exec_op()) lustre-OST0000: opcode 8: before 511 < left 544, rollback = 9 [ 1687.127819] Lustre: 59261:0:(osd_handler.c:2090:osd_trans_dump_creds()) create: 32/128/4, destroy: 0/0/0 [ 1687.131924] Lustre: 59261:0:(osd_handler.c:2097:osd_trans_dump_creds()) attr_set: 0/0/0, xattr_set: 0/0/0 [ 1687.140507] Lustre: 59261:0:(osd_handler.c:2107:osd_trans_dump_creds()) write: 2/18/1, punch: 0/0/0, quota 1/3/0 [ 1687.150578] Lustre: 59261:0:(osd_handler.c:2114:osd_trans_dump_creds()) insert: 32/544/0, delete: 0/0/0 [ 1687.158039] Lustre: 59261:0:(osd_handler.c:2121:osd_trans_dump_creds()) ref_add: 0/0/0, ref_del: 0/0/0 [ 1691.623677] Lustre: DEBUG MARKER: oleg158-server.virtnet: executing set_default_debug -1 all [ 1701.013775] Lustre: DEBUG MARKER: oleg158-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 1707.147920] Lustre: DEBUG MARKER: check files after expanding the MDT and OST filesystems [ 1714.546543] Lustre: DEBUG MARKER: create more files after expanding the MDT and OST filesystems [ 1748.449418] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 1748.456076] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 1748.517595] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 1752.576576] Lustre: server umount lustre-OST0000 complete [ 1756.305201] Lustre: server umount lustre-MDT0000 complete [ 1767.059085] Lustre: DEBUG MARKER: oleg158-server.virtnet: executing unload_modules_local [ 1769.549833] Key type lgssc unregistered [ 1769.930618] LNet: 60899:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 1769.940615] LNetError: 60899:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 1769.972700] LNet: Removed LNI 192.168.201.158@tcp [ 1770.688344] Key type .llcrypt unregistered [ 1770.693784] Key type ._llcrypt unregistered [ 1798.745671] Key type ._llcrypt registered [ 1798.750744] Key type .llcrypt registered [ 1798.881233] Lustre: DEBUG MARKER: oleg158-server.virtnet: executing load_modules_local [ 1800.282973] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 1800.349226] alg: No test for adler32 (adler32-zlib) [ 1801.373766] Lustre: Lustre: Build Version: 2.17.57_48_g018c4cb [ 1801.618734] LNet: Added LNI 192.168.201.158@tcp [8/256/0/180] [ 1803.263240] Key type lgssc registered [ 1804.675288] Lustre: Echo OBD driver; http://www.lustre.org/ [ 1812.315778] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 1812.340792] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1813.658843] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 1817.588772] Lustre: DEBUG MARKER: oleg158-server.virtnet: executing set_default_debug -1 all [ 1825.365686] Lustre: DEBUG MARKER: oleg158-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 1832.459721] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 1832.731411] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 1838.445374] Lustre: DEBUG MARKER: oleg158-server.virtnet: executing set_default_debug -1 all [ 1838.913057] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x240000400:140 to 0x240000400:161) [ 1838.981859] Lustre: 63740:0:(osd_internal.h:1460:osd_trans_exec_op()) lustre-OST0000: opcode 8: before 509 < left 544, rollback = 9 [ 1838.989715] Lustre: 63740:0:(osd_handler.c:2090:osd_trans_dump_creds()) create: 32/128/6, destroy: 0/0/0 [ 1838.996287] Lustre: 63740:0:(osd_handler.c:2097:osd_trans_dump_creds()) attr_set: 0/0/0, xattr_set: 0/0/0 [ 1839.005234] Lustre: 63740:0:(osd_handler.c:2107:osd_trans_dump_creds()) write: 2/18/1, punch: 0/0/0, quota 1/3/0 [ 1839.010792] Lustre: 63740:0:(osd_handler.c:2114:osd_trans_dump_creds()) insert: 32/544/0, delete: 0/0/0 [ 1839.015693] Lustre: 63740:0:(osd_handler.c:2121:osd_trans_dump_creds()) ref_add: 0/0/0, ref_del: 0/0/0 [ 1845.988951] Lustre: DEBUG MARKER: oleg158-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 1851.665216] Lustre: DEBUG MARKER: check files after shrinking the MDT and OST filesystems [ 1864.676944] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 1864.690467] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 1864.709807] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 1867.559956] Lustre: server umount lustre-OST0000 complete [ 1871.259236] Lustre: server umount lustre-MDT0000 complete [ 1884.331921] Lustre: DEBUG MARKER: oleg158-server.virtnet: executing unload_modules_local [ 1886.691569] Key type lgssc unregistered [ 1887.077374] LNet: 65299:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 1887.089570] LNetError: 65299:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 1888.169765] LNet: Removed LNI 192.168.201.158@tcp [ 1888.965466] Key type .llcrypt unregistered [ 1888.967422] Key type ._llcrypt unregistered [ 1906.991373] Key type ._llcrypt registered [ 1906.994730] Key type .llcrypt registered [ 1907.067928] Lustre: DEBUG MARKER: oleg158-server.virtnet: executing set_hostid [ 1925.154265] Lustre: DEBUG MARKER: oleg158-server.virtnet: executing load_modules_local [ 1925.928171] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 1925.958038] alg: No test for adler32 (adler32-zlib) [ 1927.106505] Lustre: Lustre: Build Version: 2.17.57_48_g018c4cb [ 1927.360029] LNet: Added LNI 192.168.201.158@tcp [8/256/0/180] [ 1929.023492] Key type lgssc registered [ 1930.461110] Lustre: Echo OBD driver; http://www.lustre.org/ [ 1966.833430] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 1966.871420] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1968.185778] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 1968.211226] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 1968.325142] Lustre: lustre-MDT0000: new disk, initializing [ 1968.404489] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 1968.433141] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 1972.593341] Lustre: DEBUG MARKER: oleg158-server.virtnet: executing set_default_debug -1 all [ 1982.304951] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1982.443908] Lustre: 68453:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 1982.502077] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [ 1982.505189] Lustre: Skipped 1 previous similar message [ 1982.604524] Lustre: lustre-MDT0001: new disk, initializing [ 1982.745774] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 1982.814371] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [ 1982.823674] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 1987.655538] Lustre: DEBUG MARKER: oleg158-server.virtnet: executing set_default_debug -1 all [ 1997.140638] Lustre: DEBUG MARKER: oleg158-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 2003.239635] Lustre: DEBUG MARKER: oleg158-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 2008.911548] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 2009.077255] Lustre: lustre-OST0000: new disk, initializing [ 2009.080638] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 2009.087244] Lustre: 70422:0:(osd_compat.c:1353:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 2009.162817] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 2014.283105] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [ 2014.295666] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 2014.422923] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 2014.708370] Lustre: DEBUG MARKER: oleg158-server.virtnet: executing set_default_debug -1 all [ 2023.498621] Lustre: DEBUG MARKER: oleg158-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 2028.177681] Lustre: DEBUG MARKER: oleg158-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 [ 2028.367410] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [ 2032.628150] Lustre: DEBUG MARKER: oleg158-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid 50 [ 2032.812619] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [ 2035.172135] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2035.173767] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 2035.176206] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 2035.194158] Lustre: Skipped 1 previous similar message [ 2035.209376] LustreError: Skipped 1 previous similar message [ 2040.293068] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 2040.304062] Lustre: Skipped 2 previous similar messages [ 2040.863938] Lustre: server umount lustre-OST0000 complete [ 2045.409879] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2045.416060] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 2045.416863] Lustre: Skipped 1 previous similar message [ 2045.442427] Lustre: Skipped 1 previous similar message [ 2050.532200] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 2050.544193] Lustre: Skipped 1 previous similar message [ 2051.533348] Lustre: server umount lustre-MDT0000 complete [ 2055.678600] LustreError: 68445:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1787795062 with bad export cookie 18287381690075120606 [ 2055.681779] LustreError: MGC192.168.201.158@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 2055.693578] LustreError: 68445:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 1 previous similar message [ 2056.048589] Lustre: server umount lustre-MDT0001 complete [ 2068.650590] Lustre: DEBUG MARKER: == conf-sanity test 79: format MDT/OST without mgs option (should return errors) ========================================================== 21:44:33 (1787795073) [ 2085.896647] Lustre: DEBUG MARKER: oleg158-server.virtnet: executing load_modules_local [ 2124.611391] Lustre: DEBUG MARKER: oleg158-server.virtnet: executing set_hostid [ 2136.979465] Lustre: DEBUG MARKER: oleg158-server.virtnet: executing load_modules_local [ 2175.619967] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 2175.869667] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 2175.913584] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 2175.954629] Lustre: lustre-MDT0000: new disk, initializing [ 2175.998426] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 2176.007439] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 2181.074548] Lustre: DEBUG MARKER: oleg158-server.virtnet: executing set_default_debug -1 all [ 2194.396097] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 2194.524256] Lustre: 76462:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 2194.566447] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [ 2194.569993] Lustre: Skipped 1 previous similar message [ 2194.627822] Lustre: lustre-MDT0001: new disk, initializing [ 2194.720186] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 2194.743211] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [ 2194.753432] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 2199.270556] Lustre: DEBUG MARKER: oleg158-server.virtnet: executing set_default_debug -1 all [ 2208.355423] Lustre: DEBUG MARKER: oleg158-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 2214.398365] Lustre: DEBUG MARKER: oleg158-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 2220.952864] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 2221.363786] Lustre: lustre-OST0000: new disk, initializing [ 2221.370377] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 2221.380655] Lustre: 78433:0:(osd_compat.c:1353:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 2221.514762] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 2227.223477] Lustre: DEBUG MARKER: oleg158-server.virtnet: executing set_default_debug -1 all [ 2229.386165] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [ 2229.401147] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 2229.650609] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 2238.967523] Lustre: DEBUG MARKER: oleg158-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 2244.038671] Lustre: DEBUG MARKER: oleg158-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 [ 2244.295160] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [ 2250.928500] Lustre: DEBUG MARKER: oleg158-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid 50 [ 2251.461868] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [ 2255.331163] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 2255.336934] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2255.347880] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 2260.459726] LustreError: 78417:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-OST0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 2260.470066] LustreError: 78417:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 3 previous similar messages [ 2260.532201] Lustre: server umount lustre-OST0000 complete [ 2265.578193] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2265.585416] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 2265.595785] Lustre: Skipped 2 previous similar messages [ 2265.607789] Lustre: Skipped 2 previous similar messages [ 2270.688567] LustreError: 76475:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 2270.699891] LustreError: 76475:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 1 previous similar message [ 2270.868802] Lustre: server umount lustre-MDT0000 complete [ 2274.389568] LustreError: 78432:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1787795281 with bad export cookie 18287381690075121488 [ 2274.402679] LustreError: MGC192.168.201.158@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 2274.404833] LustreError: 78432:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 6 previous similar messages [ 2274.659861] Lustre: server umount lustre-MDT0001 complete [ 2284.178921] Lustre: DEBUG MARKER: == conf-sanity test 80: mgc import reconnect race ======== 21:48:09 (1787795289) [ 2290.842233] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 2291.416882] LustreError: 80844:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 2291.445066] LustreError: 80844:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 1 previous similar message [ 2291.488194] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 2296.024089] Lustre: DEBUG MARKER: oleg158-server.virtnet: executing set_default_debug -1 all [ 2296.801609] LustreError: 80845:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 2301.921963] LustreError: 80844:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 2304.121653] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 2308.389254] Lustre: DEBUG MARKER: oleg158-server.virtnet: executing set_default_debug -1 all [ 2316.490505] Lustre: DEBUG MARKER: oleg158-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 2322.421819] Lustre: DEBUG MARKER: oleg158-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 2330.303487] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 2330.697230] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 2330.701201] Lustre: Skipped 1 previous similar message [ 2338.100340] Lustre: DEBUG MARKER: oleg158-server.virtnet: executing set_default_debug -1 all [ 2346.717218] Lustre: DEBUG MARKER: oleg158-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 2350.031748] Lustre: 83517:0:(genops.c:1774:obd_export_evict_by_uuid()) MGS: evicting c081e68f-8699-4db0-a7c5-c628af0f62e2 at adminstrative request [ 2354.144336] LustreError: MGC192.168.201.158@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 2354.170289] Lustre: *** cfs_fail_loc=906, val=2147483648*** [ 2364.391746] Lustre: MGS: Client c081e68f-8699-4db0-a7c5-c628af0f62e2 (at 0@lo) reconnecting [ 2364.422875] Lustre: Evicted from MGS (at 0@lo) after server handle changed from 0xfdc9d4c1a4e90ac2 to 0xfdc9d4c1a4e90d62 [ 2364.468712] Lustre: MGC192.168.201.158@tcp: Connection restored to 0@lo (at 0@lo) [ 2387.663669] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 2387.826372] Lustre: lustre-OST0001: new disk, initializing [ 2387.829589] Lustre: srv-lustre-OST0001: No data found on store. Initialize space. [ 2387.834567] Lustre: 83909:0:(osd_compat.c:1353:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 2395.193944] Lustre: DEBUG MARKER: oleg158-server.virtnet: executing set_default_debug -1 all [ 2397.305953] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x00000002c0000400-0x0000000300000400]:1:ost [ 2397.314432] Lustre: cli-lustre-OST0001-super: Allocated super-sequence [0x00000002c0000400-0x0000000300000400]:1:ost] [ 2397.409165] Lustre: lustre-OST0001-osc-MDT0000: update sequence from 0x100010000 to 0x2c0000401 [ 2405.319336] Lustre: DEBUG MARKER: oleg158-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0001-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 2413.025980] LustreError: lustre-OST0001-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 2413.036440] LustreError: Skipped 1 previous similar message [ 2413.040927] Lustre: lustre-OST0001-osc-MDT0000: Connection to lustre-OST0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2413.054661] Lustre: lustre-OST0001: Not available for connect from 0@lo (stopping) [ 2413.057104] Lustre: Skipped 1 previous similar message [ 2413.066400] Lustre: Skipped 1 previous similar message [ 2414.841810] Lustre: server umount lustre-OST0001 complete [ 2418.147626] LustreError: 82666:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 2418.171305] LustreError: 82666:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 1 previous similar message [ 2420.705959] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 2420.719572] LustreError: Skipped 1 previous similar message [ 2423.274468] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2423.299848] Lustre: Skipped 1 previous similar message [ 2425.271042] Lustre: server umount lustre-OST0000 complete [ 2432.485344] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 2436.447263] Lustre: server umount lustre-MDT0000 complete [ 2438.627742] LustreError: 81897:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 2438.649885] LustreError: 81897:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 3 previous similar messages [ 2439.928588] LustreError: 80825:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1787795446 with bad export cookie 18287381690075123042 [ 2439.935722] LustreError: MGC192.168.201.158@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 2439.955658] LustreError: 80825:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 1 previous similar message [ 2440.395670] Lustre: server umount lustre-MDT0001 complete [ 2449.541863] Lustre: DEBUG MARKER: == conf-sanity test 81: sparse OST indexing ============== 21:50:54 (1787795454) [ 2451.508196] Lustre: DEBUG MARKER: SKIP: conf-sanity test_81 needs >= 3 OSTs [ 2453.761969] Lustre: DEBUG MARKER: == conf-sanity test 82a: specify OSTs for file (succeed) or directory (succeed) ========================================================== 21:50:58 (1787795458) [ 2455.846590] Lustre: DEBUG MARKER: SKIP: conf-sanity test_82a needs >= 3 OSTs [ 2458.226032] Lustre: DEBUG MARKER: == conf-sanity test 82b: specify OSTs for file with --pool and --ost-list options ========================================================== 21:51:03 (1787795463) [ 2460.492210] Lustre: DEBUG MARKER: SKIP: conf-sanity test_82b needs >= 4 OSTs [ 2463.273961] Lustre: DEBUG MARKER: == conf-sanity test 82c: specify sparse OSTs for setstripe (should not crash) ========================================================== 21:51:08 (1787795468) [ 2465.415120] Lustre: DEBUG MARKER: SKIP: conf-sanity test_82c needs >= 4 OSTs [ 2467.378933] Lustre: DEBUG MARKER: == conf-sanity test 83: ENOSPACE on OST doesn't cause message VFS: Busy inodes after unmount ... ========================================================== 21:51:12 (1787795472) [ 2469.229574] Lustre: DEBUG MARKER: mount the OST /dev/mapper/ost1_flakey as a ldiskfs filesystem [ 2476.922747] loop: module loaded [ 2476.950070] loop0: detected capacity change from 0 to 4294967296 [ 2477.004692] LDISKFS-fs (loop0): mounted filesystem with ordered data mode. Opts: (null) [ 2478.655246] Lustre: DEBUG MARKER: run llverfs in partial mode on the OST ldiskfs /mnt/lustre-ost1 [ 2482.090441] Lustre: DEBUG MARKER: oleg158-server.virtnet: executing run_llverfs /mnt/lustre-ost1 -vpl no [ 2485.103305] Lustre: DEBUG MARKER: unmount the OST /dev/mapper/ost1_flakey [ 2510.463911] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 2510.518773] LustreError: 88034:0:(obd_config.c:845:class_setup()) setup lustre-OST0000-osd failed (-28) [ 2510.522755] LustreError: 88034:0:(obd_mount.c:259:lustre_start_simple()) lustre-OST0000-osd setup error -28 [ 2510.531735] LustreError: 88034:0:(tgt_mount.c:2579:server_fill_super()) Unable to start osd on /dev/mapper/ost1_flakey: -28 [ 2510.535700] LustreError: 88034:0:(super25.c:179:lustre_fill_super()) llite: Unable to mount : rc = -28 [ 2528.836191] Lustre: DEBUG MARKER: oleg158-server.virtnet: executing set_hostid [ 2541.271491] Lustre: DEBUG MARKER: oleg158-server.virtnet: executing load_modules_local [ 2578.598706] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 2578.874772] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 2578.917709] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 2579.016349] Lustre: lustre-MDT0000: new disk, initializing [ 2579.095700] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 2579.101427] Lustre: Skipped 1 previous similar message [ 2579.116121] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 2583.188419] Lustre: DEBUG MARKER: oleg158-server.virtnet: executing set_default_debug -1 all [ 2593.958526] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 2594.130501] Lustre: 90898:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 2594.350975] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 2599.550147] Lustre: DEBUG MARKER: oleg158-server.virtnet: executing set_default_debug -1 all [ 2609.681160] Lustre: DEBUG MARKER: oleg158-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 2615.132267] Lustre: DEBUG MARKER: oleg158-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 2621.414384] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 2621.705242] Lustre: 92868:0:(osd_compat.c:1353:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 2627.210839] Lustre: DEBUG MARKER: oleg158-server.virtnet: executing set_default_debug -1 all [ 2627.655157] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 2627.756392] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 2635.516708] Lustre: DEBUG MARKER: oleg158-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 2639.986440] Lustre: DEBUG MARKER: oleg158-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 [ 2640.147940] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [ 2644.782955] Lustre: DEBUG MARKER: oleg158-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid 50 [ 2645.013924] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [ 2648.545942] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2648.550104] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 2648.551781] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 2648.558156] Lustre: Skipped 3 previous similar messages [ 2648.561864] Lustre: Skipped 6 previous similar messages [ 2660.831146] Lustre: lustre-OST0000 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 2661.022495] Lustre: server umount lustre-OST0000 complete [ 2665.952362] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 2679.265640] Lustre: lustre-MDT0000 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 2679.440383] Lustre: server umount lustre-MDT0000 complete [ 2682.712834] LustreError: 90892:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1787795689 with bad export cookie 18287381690075123658 [ 2682.719946] LustreError: 90892:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 1 previous similar message [ 2682.722323] LustreError: MGC192.168.201.158@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 2682.920352] Lustre: server umount lustre-MDT0001 complete [ 2691.161182] Lustre: DEBUG MARKER: == conf-sanity test 84: check recovery_hard_time ========= 21:54:56 (1787795696) [ 2697.596360] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 2698.053242] LustreError: 95283:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 2702.514144] Lustre: DEBUG MARKER: oleg158-server.virtnet: executing set_default_debug -1 all [ 2711.155621] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 2711.630127] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-60 [ 2711.647041] Lustre: Skipped 3 previous similar messages [ 2717.201144] Lustre: DEBUG MARKER: oleg158-server.virtnet: executing set_default_debug -1 all [ 2725.128865] Lustre: DEBUG MARKER: oleg158-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 2730.790748] Lustre: DEBUG MARKER: oleg158-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 2737.455840] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 2744.358806] Lustre: DEBUG MARKER: oleg158-server.virtnet: executing set_default_debug -1 all [ 2752.114449] Lustre: DEBUG MARKER: oleg158-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 2756.523389] Lustre: DEBUG MARKER: oleg158-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 [ 2756.777423] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [ 2762.047623] Lustre: DEBUG MARKER: oleg158-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid 50 [ 2762.366895] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [ 2769.223974] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 2769.346644] Lustre: lustre-OST0001: new disk, initializing [ 2769.349962] Lustre: Skipped 2 previous similar messages [ 2769.353961] Lustre: srv-lustre-OST0001: No data found on store. Initialize space. [ 2769.358187] Lustre: Skipped 3 previous similar messages [ 2769.363659] Lustre: 98652:0:(osd_compat.c:1353:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 2775.878848] Lustre: DEBUG MARKER: oleg158-server.virtnet: executing set_default_debug -1 all [ 2777.184935] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x00000002c0000400-0x0000000300000400]:1:ost [ 2777.189841] Lustre: Skipped 2 previous similar messages [ 2777.194939] Lustre: cli-lustre-OST0001-super: Allocated super-sequence [0x00000002c0000400-0x0000000300000400]:1:ost] [ 2777.284504] Lustre: lustre-OST0001-osc-MDT0000: update sequence from 0x100010000 to 0x2c0000401 [ 2784.853255] Lustre: DEBUG MARKER: oleg158-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0001-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 2789.727441] Lustre: DEBUG MARKER: oleg158-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0001-osc-MDT0000.ost_server_uuid 50 [ 2790.002661] Lustre: DEBUG MARKER: os[cp].lustre-OST0001-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [ 2794.653679] Lustre: DEBUG MARKER: oleg158-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0001-osc-MDT0001.ost_server_uuid 50 [ 2794.840784] Lustre: DEBUG MARKER: os[cp].lustre-OST0001-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [ 2814.225091] Lustre: DEBUG MARKER: mds1 REPLAY BARRIER on lustre-MDT0000 [ 2847.507585] Lustre: Failing over lustre-MDT0000 [ 2848.008581] Lustre: server umount lustre-MDT0000 complete [ 2848.537211] LustreError: 95278:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0000: not available for connect from 192.168.201.58@tcp (no target). If you are running an HA pair check that the target is mounted on the other server. [ 2848.556728] LustreError: 95278:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 3 previous similar messages [ 2849.254462] Lustre: lustre-MDT0000-lwp-OST0000: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2849.292488] Lustre: Skipped 4 previous similar messages [ 2865.635630] Lustre: 66675:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1787795856/real 1787795856] req@000000005a851b7c x1874638860988928/t0(0) o400->MGC192.168.201.158@tcp@0@lo:26/25 lens 224/224 e 0 to 1 dl 1787795872 ref 1 fl Rpc:XNQr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 [ 2865.685537] LustreError: MGC192.168.201.158@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 2874.855109] LDISKFS-fs (dm-0): 4 truncates cleaned up [ 2874.859572] LDISKFS-fs (dm-0): recovery complete [ 2874.866659] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 2875.880156] LustreError: 66673:0:(client.c:1404:ptlrpc_import_delay_req()) @@@ invalidate in flight req@000000008f675194 x1874638860998272/t0(0) o250->MGC192.168.201.158@tcp@0@lo:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl Rpc:NQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 [ 2876.358460] Lustre: lustre-MDT0000: in recovery but waiting for the first client to connect [ 2879.238221] Lustre: lustre-MDT0000: Will be in recovery for at least 1:00, or until 3 clients reconnect [ 2879.239738] Lustre: 98662:0:(ldlm_lib.c:2110:extend_recovery_timer()) lustre-MDT0000: extended recovery timer reached hard limit: 60, extend: 0 [ 2881.506866] Lustre: 95284:0:(ldlm_lib.c:2110:extend_recovery_timer()) lustre-MDT0000: extended recovery timer reached hard limit: 60, extend: 0 [ 2881.521277] Lustre: 95284:0:(ldlm_lib.c:2110:extend_recovery_timer()) Skipped 1 previous similar message [ 2881.554968] Lustre: lustre-MDT0000-lwp-MDT0001: Connection restored to 0@lo (at 0@lo) [ 2881.646591] LustreError: 101505:0:(ldlm_lib.c:2723:replay_request_or_update()) cfs_fail_timeout id 709 sleeping for 300ms [ 2881.776814] Lustre: DEBUG MARKER: oleg158-server.virtnet: executing set_default_debug -1 all [ 2881.984803] LustreError: 101505:0:(ldlm_lib.c:2723:replay_request_or_update()) cfs_fail_timeout id 709 awake [ 2882.329307] LustreError: 101505:0:(ldlm_lib.c:2723:replay_request_or_update()) cfs_fail_timeout id 709 sleeping for 300ms [ 2882.350159] LustreError: 101505:0:(ldlm_lib.c:2723:replay_request_or_update()) Skipped 1 previous similar message [ 2882.688275] LustreError: 101505:0:(ldlm_lib.c:2723:replay_request_or_update()) cfs_fail_timeout id 709 awake [ 2882.703507] LustreError: 101505:0:(ldlm_lib.c:2723:replay_request_or_update()) Skipped 1 previous similar message [ 2882.718764] Lustre: 101505:0:(ldlm_lib.c:2110:extend_recovery_timer()) lustre-MDT0000: extended recovery timer reached hard limit: 60, extend: 1 [ 2882.736664] Lustre: 101505:0:(ldlm_lib.c:2110:extend_recovery_timer()) Skipped 7 previous similar messages [ 2883.380253] LustreError: 101505:0:(ldlm_lib.c:2723:replay_request_or_update()) cfs_fail_timeout id 709 sleeping for 300ms [ 2883.390289] LustreError: 101505:0:(ldlm_lib.c:2723:replay_request_or_update()) Skipped 2 previous similar messages [ 2883.727320] LustreError: 101505:0:(ldlm_lib.c:2723:replay_request_or_update()) cfs_fail_timeout id 709 awake [ 2883.738905] LustreError: 101505:0:(ldlm_lib.c:2723:replay_request_or_update()) Skipped 2 previous similar messages [ 2885.026632] Lustre: 101505:0:(ldlm_lib.c:2110:extend_recovery_timer()) lustre-MDT0000: extended recovery timer reached hard limit: 60, extend: 1 [ 2885.065988] Lustre: 101505:0:(ldlm_lib.c:2110:extend_recovery_timer()) Skipped 6 previous similar messages [ 2885.392804] LustreError: 101505:0:(ldlm_lib.c:2723:replay_request_or_update()) cfs_fail_timeout id 709 sleeping for 300ms [ 2885.401367] LustreError: 101505:0:(ldlm_lib.c:2723:replay_request_or_update()) Skipped 5 previous similar messages [ 2886.047190] LustreError: 101505:0:(ldlm_lib.c:2723:replay_request_or_update()) cfs_fail_timeout id 709 awake [ 2886.059829] LustreError: 101505:0:(ldlm_lib.c:2723:replay_request_or_update()) Skipped 6 previous similar messages [ 2889.256220] Lustre: 101505:0:(ldlm_lib.c:2110:extend_recovery_timer()) lustre-MDT0000: extended recovery timer reached hard limit: 60, extend: 1 [ 2889.265804] Lustre: 101505:0:(ldlm_lib.c:2110:extend_recovery_timer()) Skipped 12 previous similar messages [ 2889.595648] LustreError: 101505:0:(ldlm_lib.c:2723:replay_request_or_update()) cfs_fail_timeout id 709 sleeping for 300ms [ 2889.612219] LustreError: 101505:0:(ldlm_lib.c:2723:replay_request_or_update()) Skipped 12 previous similar messages [ 2890.247154] LustreError: 101505:0:(ldlm_lib.c:2723:replay_request_or_update()) cfs_fail_timeout id 709 awake [ 2890.257373] LustreError: 101505:0:(ldlm_lib.c:2723:replay_request_or_update()) Skipped 12 previous similar messages [ 2897.560473] Lustre: 101505:0:(ldlm_lib.c:2110:extend_recovery_timer()) lustre-MDT0000: extended recovery timer reached hard limit: 60, extend: 1 [ 2897.580385] Lustre: 101505:0:(ldlm_lib.c:2110:extend_recovery_timer()) Skipped 25 previous similar messages [ 2897.915117] LustreError: 101505:0:(ldlm_lib.c:2723:replay_request_or_update()) cfs_fail_timeout id 709 sleeping for 300ms [ 2897.930666] LustreError: 101505:0:(ldlm_lib.c:2723:replay_request_or_update()) Skipped 25 previous similar messages [ 2898.559912] LustreError: 101505:0:(ldlm_lib.c:2723:replay_request_or_update()) cfs_fail_timeout id 709 awake [ 2898.563249] LustreError: 101505:0:(ldlm_lib.c:2723:replay_request_or_update()) Skipped 25 previous similar messages [ 2913.584064] Lustre: 101505:0:(ldlm_lib.c:2110:extend_recovery_timer()) lustre-MDT0000: extended recovery timer reached hard limit: 60, extend: 1 [ 2913.597688] Lustre: 101505:0:(ldlm_lib.c:2110:extend_recovery_timer()) Skipped 49 previous similar messages [ 2913.954567] LustreError: 101505:0:(ldlm_lib.c:2723:replay_request_or_update()) cfs_fail_timeout id 709 sleeping for 300ms [ 2913.967731] LustreError: 101505:0:(ldlm_lib.c:2723:replay_request_or_update()) Skipped 49 previous similar messages [ 2914.623239] LustreError: 101505:0:(ldlm_lib.c:2723:replay_request_or_update()) cfs_fail_timeout id 709 awake [ 2914.629195] LustreError: 101505:0:(ldlm_lib.c:2723:replay_request_or_update()) Skipped 49 previous similar messages [ 2939.215859] Lustre: 101505:0:(ldlm_lib.c:2419:target_recovery_overseer()) lustre-MDT0000 recovery is aborted by hard timeout [ 2939.226603] Lustre: 101505:0:(ldlm_lib.c:2429:target_recovery_overseer()) recovery is aborted, evict exports in recovery [ 2939.234176] Lustre: 101505:0:(genops.c:1601:class_disconnect_stale_exports()) lustre-MDT0000: disconnect stale client 249e851a-6bb4-4950-9013-44f17f293665@192.168.201.58@tcp [ 2939.256414] Lustre: lustre-MDT0000: disconnecting 1 stale clients [ 2939.261238] Lustre: 101505:0:(ldlm_lib.c:1939:abort_req_replay_queue()) @@@ aborted: req@00000000a601abf3 x1874638842958848/t0(8589934776) o35->249e851a-6bb4-4950-9013-44f17f293665@192.168.201.58@tcp:501/0 lens 392/0 e 0 to 0 dl 1787795956 ref 1 fl Complete:/604/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:0 [ 2939.313488] Lustre: lustre-MDT0000-osd: cancel update llog [0x200000400:0x1:0x0] [ 2939.318210] Lustre: lustre-MDT0000: Denying connection for new client 249e851a-6bb4-4950-9013-44f17f293665 (at 192.168.201.58@tcp), waiting for 3 known clients (2 recovered, 0 in progress, and 1 evicted) to recover in 0:00 [ 2939.345063] Lustre: lustre-MDT0001-osp-MDT0000: cancel update llog [0x240000401:0x1:0x0] [ 2939.379964] Lustre: lustre-MDT0000-osp-MDT0001: Connection restored to 0@lo (at 0@lo) [ 2939.383145] Lustre: Skipped 2 previous similar messages [ 2939.386594] Lustre: lustre-MDT0000: Recovery over after 1:00, of 3 clients 2 recovered and 1 was evicted. [ 2939.431326] Lustre: lustre-OST0001: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x2c0000401:47 to 0x2c0000401:545) [ 2939.434846] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:79 to 0x280000401:577) [ 2953.194886] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2953.195954] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 2953.202668] Lustre: Skipped 2 previous similar messages [ 2953.212130] Lustre: Skipped 13 previous similar messages [ 2957.908634] Lustre: server umount lustre-OST0000 complete [ 2961.900899] LustreError: lustre-OST0001-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 2967.546752] Lustre: server umount lustre-OST0001 complete [ 2977.648761] Lustre: server umount lustre-MDT0000 complete [ 2978.784550] LustreError: 95278:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 2978.804585] LustreError: 95278:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 45 previous similar messages [ 2982.043246] LustreError: 95265:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1787795989 with bad export cookie 18287381690075174870 [ 2982.054033] LustreError: MGC192.168.201.158@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 2982.063126] LustreError: 95265:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 5 previous similar messages [ 2982.787426] Lustre: server umount lustre-MDT0001 complete [ 2993.656265] Lustre: DEBUG MARKER: == conf-sanity test 85: osd_ost init: fail ea_fid_set ==== 21:59:59 (1787795999) [ 3000.431050] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 3000.446205] Lustre: *** cfs_fail_loc=197, val=0*** [ 3016.031823] LustreError: Failed to get MGS log lustre-sptlrpc and no local copy. [ 3021.151625] LustreError: 103484:0:(mgc_request.c:233:do_config_log_add()) MGC192.168.201.158@tcp: failed processing log, type 4: rc = -110 [ 3026.300880] Lustre: *** cfs_fail_loc=197, val=0*** [ 3026.304815] Lustre: Skipped 65 previous similar messages [ 3046.879592] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 3046.890891] Lustre: Skipped 3 previous similar messages [ 3054.901954] Lustre: DEBUG MARKER: oleg158-server.virtnet: executing set_default_debug -1 all [ 3064.338305] Lustre: DEBUG MARKER: oleg158-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 3066.769526] Lustre: server umount lustre-OST0000 complete [ 3077.509028] Lustre: DEBUG MARKER: == conf-sanity test 86: Replacing mkfs.lustre -G option == 22:01:22 (1787796082) [ 3100.450037] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 3105.702606] Lustre: DEBUG MARKER: oleg158-server.virtnet: executing set_default_debug -1 all [ 3116.226771] Lustre: DEBUG MARKER: oleg158-client.virtnet: executing wait_import_state_mount (FULL|IDLE) mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 3120.535142] LustreError: 105149:0:(osp_object.c:618:osp_attr_get()) lustre-MDT0001-osp-MDT0000: osp_attr_get update error [0x200000009:0x1:0x0]: rc = -5 [ 3120.546248] LustreError: 105149:0:(lod_sub_object.c:917:lod_sub_prep_llog()) lustre-MDT0000-mdtlov: can't get id from catalogs: rc = -5 [ 3120.552392] LustreError: 105149:0:(lod_dev.c:511:lod_sub_recovery_thread()) lustre-MDT0001-osp-MDT0000: get update log duration 20, retries 0, failed: rc = -5 [ 3120.781031] Lustre: server umount lustre-MDT0000 complete [ 3158.306903] Lustre: DEBUG MARKER: oleg158-server.virtnet: executing set_hostid [ 3171.701939] Lustre: DEBUG MARKER: oleg158-server.virtnet: executing load_modules_local [ 3210.739763] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 3211.123119] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 3211.160261] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 3211.238238] Lustre: lustre-MDT0000: new disk, initializing [ 3211.393776] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 3215.758829] Lustre: DEBUG MARKER: oleg158-server.virtnet: executing set_default_debug -1 all [ 3226.480335] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 3226.703975] Lustre: 109324:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 3226.991236] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 3231.457640] Lustre: DEBUG MARKER: oleg158-server.virtnet: executing set_default_debug -1 all [ 3242.879299] Lustre: DEBUG MARKER: oleg158-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 3249.954514] Lustre: DEBUG MARKER: oleg158-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 3258.716920] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 3259.167945] Lustre: 111295:0:(osd_compat.c:1353:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 3265.826490] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 3266.251571] Lustre: DEBUG MARKER: oleg158-server.virtnet: executing set_default_debug -1 all [ 3275.435658] Lustre: DEBUG MARKER: oleg158-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 3281.225160] Lustre: DEBUG MARKER: oleg158-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 [ 3281.497466] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [ 3287.110289] Lustre: DEBUG MARKER: oleg158-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid 50 [ 3287.417989] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [ 3291.615975] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 3291.619830] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 3291.633829] LustreError: Skipped 1 previous similar message [ 3291.634690] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 3291.634695] Lustre: Skipped 5 previous similar messages [ 3291.674110] Lustre: Skipped 5 previous similar messages [ 3295.982383] Lustre: server umount lustre-OST0000 complete [ 3305.812655] Lustre: server umount lustre-MDT0000 complete [ 3306.980146] LustreError: 109338:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 3307.008063] LustreError: 109338:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 5 previous similar messages [ 3309.717485] LustreError: 109317:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1787796316 with bad export cookie 18287381690075185510 [ 3309.719772] LustreError: MGC192.168.201.158@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 3309.728709] LustreError: 109317:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 1 previous similar message [ 3310.329297] Lustre: server umount lustre-MDT0001 complete [ 3318.976688] Lustre: DEBUG MARKER: == conf-sanity test 87: check if MDT inode can hold EAs with N stripes properly ========================================================== 22:05:24 (1787796324) [ 3329.972860] Lustre: DEBUG MARKER: oleg158-server.virtnet: executing unload_modules_local [ 3333.242321] Key type lgssc unregistered [ 3333.526853] LNet: 113787:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 3333.548536] LNetError: 113787:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 3333.566887] LNet: Removed LNI 192.168.201.158@tcp [ 3334.744661] Key type .llcrypt unregistered [ 3334.748515] Key type ._llcrypt unregistered [ 3354.216309] Key type ._llcrypt registered [ 3354.217914] Key type .llcrypt registered [ 3354.397111] Lustre: DEBUG MARKER: oleg158-server.virtnet: executing set_hostid [ 3370.805570] Lustre: DEBUG MARKER: oleg158-server.virtnet: executing load_modules_local [ 3371.980156] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 3372.004719] alg: No test for adler32 (adler32-zlib) [ 3373.146503] Lustre: Lustre: Build Version: 2.17.57_48_g018c4cb [ 3373.450691] LNet: Added LNI 192.168.201.158@tcp [8/256/0/180] [ 3375.143134] Key type lgssc registered [ 3376.334859] Lustre: Echo OBD driver; http://www.lustre.org/ [ 3422.460721] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 3422.478256] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 3423.742699] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 3423.767357] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 3423.817641] Lustre: lustre-MDT0000: new disk, initializing [ 3423.907734] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 3423.920696] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 3428.333943] Lustre: DEBUG MARKER: oleg158-server.virtnet: executing set_default_debug -1 all [ 3439.799573] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 3440.089168] Lustre: lustre-OST0000: new disk, initializing [ 3440.092439] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 3440.100602] Lustre: Skipped 1 previous similar message [ 3440.106555] Lustre: 117960:0:(osd_compat.c:1353:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 3440.207792] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 3446.710957] Lustre: DEBUG MARKER: oleg158-server.virtnet: executing set_default_debug -1 all [ 3448.428369] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:0:ost [ 3448.441501] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:0:ost] [ 3448.474948] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x240000400 [ 3458.336513] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 3458.540970] Lustre: lustre-OST0001: new disk, initializing [ 3458.543831] Lustre: srv-lustre-OST0001: No data found on store. Initialize space. [ 3458.548643] Lustre: 118823:0:(osd_compat.c:1353:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 3458.641417] Lustre: lustre-OST0001: Imperative Recovery not enabled, recovery window 60-180 [ 3465.948178] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:1:ost [ 3465.954723] Lustre: cli-lustre-OST0001-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:1:ost] [ 3465.975263] Lustre: lustre-OST0001-osc-MDT0000: update sequence from 0x100010000 to 0x280000400 [ 3466.408702] Lustre: DEBUG MARKER: oleg158-server.virtnet: executing set_default_debug -1 all [ 3476.480964] Lustre: lustre-MDT0000-lwp-OST0000: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 3476.495395] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 3476.506859] Lustre: Skipped 1 previous similar message [ 3476.524503] Lustre: Skipped 1 previous similar message [ 3481.569594] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 3481.573825] Lustre: Skipped 1 previous similar message [ 3482.292131] Lustre: server umount lustre-MDT0000 complete [ 3491.220186] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: (null) [ 3495.455303] LustreError: 118822:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1787796502 with bad export cookie 9690823884340023742 [ 3495.464161] LustreError: MGC192.168.201.158@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 3495.469032] LustreError: 118822:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 1 previous similar message [ 3495.526339] Lustre: server umount lustre-OST0000 complete [ 3499.104719] Lustre: server umount lustre-OST0001 complete [ 3508.382143] Lustre: DEBUG MARKER: == conf-sanity test 88a: tunefs.lustre persists a --mountfsoptions-only change ========================================================== 22:08:33 (1787796513) [ 3535.701413] Lustre: DEBUG MARKER: == conf-sanity test 88b: check the default mount options can be overridden ========================================================== 22:09:00 (1787796540) [ 3548.015400] Lustre: DEBUG MARKER: oleg158-server.virtnet: executing load_modules_local [ 3572.145389] Lustre: DEBUG MARKER: == conf-sanity test 89: check tunefs --param and --erase-param{s} options ========================================================== 22:09:37 (1787796577) [ 3610.517961] Lustre: DEBUG MARKER: oleg158-server.virtnet: executing set_hostid [ 3620.420992] Lustre: DEBUG MARKER: oleg158-server.virtnet: executing load_modules_local [ 3657.936067] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 3658.254796] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 3658.313526] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 3658.380387] Lustre: lustre-MDT0000: new disk, initializing [ 3658.452779] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 3658.474444] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 3663.523640] Lustre: DEBUG MARKER: oleg158-server.virtnet: executing set_default_debug -1 all [ 3673.811754] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 3673.930429] Lustre: 126580:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 3673.990413] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [ 3673.995618] Lustre: Skipped 1 previous similar message [ 3674.084652] Lustre: lustre-MDT0001: new disk, initializing [ 3674.194324] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 3674.245663] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [ 3674.264022] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 3679.381747] Lustre: DEBUG MARKER: oleg158-server.virtnet: executing set_default_debug -1 all [ 3690.942916] Lustre: DEBUG MARKER: oleg158-client.virtnet: executing wait_import_state_mount FULL failover.node=192.0.2.254@tcp0,mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 3698.899203] Lustre: DEBUG MARKER: oleg158-client.virtnet: executing wait_import_state_mount FULL failover.node=192.0.2.254@tcp0,mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 3707.659219] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 3707.866375] Lustre: lustre-OST0000: new disk, initializing [ 3707.868613] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 3707.871902] Lustre: 128551:0:(osd_compat.c:1353:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 3707.950484] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 3713.779969] Lustre: DEBUG MARKER: oleg158-server.virtnet: executing set_default_debug -1 all [ 3716.660860] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [ 3716.670975] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 3716.791928] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 3722.841971] Lustre: DEBUG MARKER: oleg158-client.virtnet: executing wait_import_state_mount (FULL|IDLE) failover.node=192.0.2.254@tcp0,osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 3728.084868] Lustre: DEBUG MARKER: oleg158-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 [ 3728.340950] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [ 3732.478139] Lustre: DEBUG MARKER: oleg158-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid 50 [ 3732.670796] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [ 3737.056663] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 3737.072627] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 3737.102664] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 3737.571646] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 3740.535797] Lustre: server umount lustre-OST0000 complete [ 3745.767211] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 3745.774946] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 3745.788479] Lustre: Skipped 1 previous similar message [ 3745.792134] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 3745.798868] Lustre: Skipped 1 previous similar message [ 3747.809378] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 3750.121807] Lustre: server umount lustre-MDT0000 complete [ 3753.253725] LustreError: 126573:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1787796760 with bad export cookie 9690823884340024967 [ 3753.264250] LustreError: MGC192.168.201.158@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 3753.265270] LustreError: 126573:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 3 previous similar messages [ 3753.699993] Lustre: server umount lustre-MDT0001 complete [ 3762.118742] Lustre: DEBUG MARKER: == conf-sanity test 90a: check max_mod_rpcs_in_flight is enforced ========================================================== 22:12:47 (1787796767) [ 3768.387774] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 3768.753631] LustreError: 130960:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 3768.764921] LustreError: 130960:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 3 previous similar messages [ 3768.831770] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 3773.073746] Lustre: DEBUG MARKER: oleg158-server.virtnet: executing set_default_debug -1 all [ 3773.924264] LustreError: 130961:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 3778.027555] LustreError: 130960:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 3780.461708] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 3784.981296] Lustre: DEBUG MARKER: oleg158-server.virtnet: executing set_default_debug -1 all [ 3793.191950] Lustre: DEBUG MARKER: oleg158-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 3799.636369] Lustre: DEBUG MARKER: oleg158-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 3807.746372] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 3808.342005] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 3808.347066] Lustre: Skipped 1 previous similar message [ 3814.223745] Lustre: DEBUG MARKER: oleg158-server.virtnet: executing set_default_debug -1 all [ 3823.025528] Lustre: DEBUG MARKER: oleg158-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 3831.289755] Lustre: *** cfs_fail_loc=159, val=0*** [ 3847.488192] Lustre: lustre-MDT0000: Client f2bf29a3-6a6b-480d-9b0e-b2519d29a389 (at 192.168.201.58@tcp) reconnecting [ 3849.962564] Lustre: *** cfs_fail_loc=159, val=0*** [ 3849.981877] Lustre: Skipped 5 previous similar messages [ 3865.434370] Lustre: lustre-MDT0000: Client f2bf29a3-6a6b-480d-9b0e-b2519d29a389 (at 192.168.201.58@tcp) reconnecting [ 3872.228224] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 3872.229831] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 3872.237287] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 3872.247775] Lustre: Skipped 1 previous similar message [ 3872.260253] Lustre: Skipped 3 previous similar messages [ 3872.627213] Lustre: server umount lustre-OST0000 complete [ 3882.233801] Lustre: server umount lustre-MDT0000 complete [ 3882.470457] LustreError: 133683:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 3882.482013] LustreError: 133683:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 1 previous similar message [ 3885.345104] LustreError: 130942:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1787796892 with bad export cookie 9690823884340025849 [ 3885.345926] LustreError: MGC192.168.201.158@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 3885.362574] LustreError: 130942:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 1 previous similar message [ 3885.662422] Lustre: server umount lustre-MDT0001 complete [ 3895.590598] Lustre: DEBUG MARKER: oleg158-server.virtnet: executing unload_modules_local [ 3898.007704] Key type lgssc unregistered [ 3898.298993] LNet: 134742:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 3898.302451] LNetError: 134742:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 3898.317269] LNet: Removed LNI 192.168.201.158@tcp [ 3899.144307] Key type .llcrypt unregistered [ 3899.146103] Key type ._llcrypt unregistered [ 3922.269248] Key type ._llcrypt registered [ 3922.273128] Key type .llcrypt registered [ 3922.377077] Lustre: DEBUG MARKER: oleg158-server.virtnet: executing load_modules_local [ 3923.492310] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 3923.531637] alg: No test for adler32 (adler32-zlib) [ 3924.772316] Lustre: Lustre: Build Version: 2.17.57_48_g018c4cb [ 3924.959936] LNet: Added LNI 192.168.201.158@tcp [8/256/0/180] [ 3926.663240] Key type lgssc registered [ 3928.156060] Lustre: Echo OBD driver; http://www.lustre.org/ [ 3936.745890] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 3936.761981] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 3938.287855] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 3943.131430] Lustre: DEBUG MARKER: oleg158-server.virtnet: executing set_default_debug -1 all [ 3948.526445] LustreError: 136156:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 3948.565913] LustreError: 136156:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 2 previous similar messages [ 3951.836968] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 3952.190523] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 3956.587261] Lustre: DEBUG MARKER: oleg158-server.virtnet: executing set_default_debug -1 all [ 3963.931474] Lustre: DEBUG MARKER: oleg158-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 3970.209289] Lustre: DEBUG MARKER: oleg158-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 3976.399142] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 3976.751042] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 3982.668430] Lustre: DEBUG MARKER: oleg158-server.virtnet: executing set_default_debug -1 all [ 3983.920536] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:42 to 0x280000401:65) [ 3990.072477] Lustre: DEBUG MARKER: oleg158-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 3999.173823] Lustre: *** cfs_fail_loc=159, val=0*** [ 4015.435769] Lustre: lustre-MDT0000: Client 38db09f2-acbf-4dda-b6e2-406131f3cfbf (at 192.168.201.58@tcp) reconnecting [ 4017.524843] Lustre: *** cfs_fail_loc=159, val=0*** [ 4033.824412] Lustre: lustre-MDT0001: Client 38db09f2-acbf-4dda-b6e2-406131f3cfbf (at 192.168.201.58@tcp) reconnecting [ 4035.202619] Lustre: *** cfs_fail_loc=159, val=0*** [ 4035.204279] Lustre: Skipped 3 previous similar messages [ 4051.800570] Lustre: lustre-MDT0001: Client 38db09f2-acbf-4dda-b6e2-406131f3cfbf (at 192.168.201.58@tcp) reconnecting [ 4067.416765] Lustre: 139212:0:(mdt_handler.c:64:mdt_max_mod_rpcs_per_client_set()) max_mod_rpcs_per_client is deprecated, set mdt.*.max_mod_rpcs_in_flight parameter instead [ 4070.015283] Lustre: *** cfs_fail_loc=159, val=0*** [ 4070.017126] Lustre: Skipped 4 previous similar messages [ 4085.611128] Lustre: lustre-MDT0000: Client 8c5f3a2b-6d0b-4272-8f8b-6a35c573232b (at 192.168.201.58@tcp) reconnecting [ 4087.709159] Lustre: *** cfs_fail_loc=159, val=0*** [ 4087.716655] Lustre: Skipped 14 previous similar messages [ 4103.498570] Lustre: lustre-MDT0000: Client 8c5f3a2b-6d0b-4272-8f8b-6a35c573232b (at 192.168.201.58@tcp) reconnecting [ 4105.075808] Lustre: 139507:0:(mdt_handler.c:64:mdt_max_mod_rpcs_per_client_set()) max_mod_rpcs_per_client is deprecated, set mdt.*.max_mod_rpcs_in_flight parameter instead [ 4115.942212] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 4115.956822] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4115.962725] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 4115.976616] Lustre: Skipped 1 previous similar message [ 4116.970142] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 4116.981901] Lustre: Skipped 2 previous similar messages [ 4119.026440] Lustre: server umount lustre-OST0000 complete [ 4126.176582] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 4126.186778] LustreError: Skipped 6 previous similar messages [ 4126.195550] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4126.228114] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 4127.209127] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4130.692994] Lustre: server umount lustre-MDT0000 complete [ 4132.328482] LustreError: 139332:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 4132.358273] LustreError: 139332:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 1 previous similar message [ 4134.662375] LustreError: 139456:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1787797141 with bad export cookie 5253900685067661976 [ 4134.676732] LustreError: MGC192.168.201.158@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 4134.686564] LustreError: 139456:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 2 previous similar messages [ 4134.964923] Lustre: server umount lustre-MDT0001 complete [ 4147.946432] Lustre: DEBUG MARKER: oleg158-server.virtnet: executing unload_modules_local [ 4151.078719] Key type lgssc unregistered [ 4151.331834] LNet: 140566:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 4151.337017] LNetError: 140566:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 4151.356223] LNet: Removed LNI 192.168.201.158@tcp [ 4152.512628] Key type .llcrypt unregistered [ 4152.518754] Key type ._llcrypt unregistered [ 4184.083309] Key type ._llcrypt registered [ 4184.085746] Key type .llcrypt registered [ 4184.257889] Lustre: DEBUG MARKER: oleg158-server.virtnet: executing load_modules_local [ 4185.297605] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 4185.359405] alg: No test for adler32 (adler32-zlib) [ 4186.499649] Lustre: Lustre: Build Version: 2.17.57_48_g018c4cb [ 4186.820190] LNet: Added LNI 192.168.201.158@tcp [8/256/0/180] [ 4188.455233] Key type lgssc registered [ 4189.550715] Lustre: Echo OBD driver; http://www.lustre.org/ [ 4198.549952] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 4198.617906] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 4200.212375] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 4204.646899] Lustre: DEBUG MARKER: oleg158-server.virtnet: executing set_default_debug -1 all [ 4210.667408] LustreError: 141981:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 4210.681097] LustreError: 141981:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 2 previous similar messages [ 4214.160555] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 4214.660456] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 4219.255707] Lustre: DEBUG MARKER: oleg158-server.virtnet: executing set_default_debug -1 all [ 4228.060448] Lustre: DEBUG MARKER: oleg158-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 4233.999373] Lustre: DEBUG MARKER: oleg158-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 4241.005872] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 4241.275703] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 4246.449212] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:85 to 0x280000401:129) [ 4250.074448] Lustre: DEBUG MARKER: oleg158-server.virtnet: executing set_default_debug -1 all [ 4258.285236] Lustre: DEBUG MARKER: oleg158-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 4259.792053] Lustre: lustre-OST0000: new connection from lustre-MDT0001-mdtlov (cleaning up unused objects from 0x280000400:72 to 0x280000400:97) [ 4264.774242] Lustre: 144565:0:(mdt_handler.c:64:mdt_max_mod_rpcs_per_client_set()) max_mod_rpcs_per_client is deprecated, set mdt.*.max_mod_rpcs_in_flight parameter instead [ 4267.319776] Lustre: 144613:0:(mdt_handler.c:64:mdt_max_mod_rpcs_per_client_set()) max_mod_rpcs_per_client is deprecated, set mdt.*.max_mod_rpcs_in_flight parameter instead [ 4268.658890] Lustre: Modifying parameter lustre.mdc.lustre-MDT0000-mdc-*.max_rpcs_in_flight=8 in log params [ 4269.305566] Lustre: Setting parameter lustre.mdc.lustre-MDT0000-mdc-*.max_mod_rpcs_in_flight=8 in log params [ 4272.395533] Lustre: 144760:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre/mdc.lustre-MDT0000-mdc-*.max_rpcs_in_flight=8 (mode = 0) failed: rc = -17 [ 4273.260077] Lustre: Modifying parameter lustre.mdc.lustre-MDT0000-mdc-*.max_mod_rpcs_in_flight=7 in log params [ 4274.303471] Lustre: 144858:0:(mdt_handler.c:64:mdt_max_mod_rpcs_per_client_set()) max_mod_rpcs_per_client is deprecated, set mdt.*.max_mod_rpcs_in_flight parameter instead [ 4280.291885] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4280.298658] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 4280.314614] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 4280.338840] Lustre: Skipped 1 previous similar message [ 4284.687752] Lustre: server umount lustre-OST0000 complete [ 4290.529855] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4290.551927] Lustre: Skipped 1 previous similar message [ 4290.560917] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 4295.139832] Lustre: server umount lustre-MDT0000 complete [ 4295.650054] LustreError: 141977:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 4295.663603] LustreError: 141977:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 1 previous similar message [ 4299.139976] LustreError: 141960:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1787797306 with bad export cookie 14545622190687512069 [ 4299.141847] LustreError: MGC192.168.201.158@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 4299.148946] LustreError: 141960:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 4299.458599] Lustre: server umount lustre-MDT0001 complete [ 4314.051048] Lustre: DEBUG MARKER: oleg158-server.virtnet: executing unload_modules_local [ 4317.141378] Key type lgssc unregistered [ 4317.399653] LNet: 145922:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 4317.409687] LNetError: 145922:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 4317.435570] LNet: Removed LNI 192.168.201.158@tcp [ 4318.412439] Key type .llcrypt unregistered [ 4318.413850] Key type ._llcrypt unregistered [ 4344.883834] Key type ._llcrypt registered [ 4344.890691] Key type .llcrypt registered [ 4345.024233] Lustre: DEBUG MARKER: oleg158-server.virtnet: executing load_modules_local [ 4345.677919] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 4345.694718] alg: No test for adler32 (adler32-zlib) [ 4346.839525] Lustre: Lustre: Build Version: 2.17.57_48_g018c4cb [ 4347.137544] LNet: Added LNI 192.168.201.158@tcp [8/256/0/180] [ 4348.936411] Key type lgssc registered [ 4350.379103] Lustre: Echo OBD driver; http://www.lustre.org/ [ 4358.968546] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 4358.986841] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 4360.660375] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 4366.341473] Lustre: DEBUG MARKER: oleg158-server.virtnet: executing set_default_debug -1 all [ 4370.933974] LustreError: 147339:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 4370.950272] LustreError: 147339:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 2 previous similar messages [ 4376.033829] LustreError: 147338:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 4376.977101] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 4377.342385] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 4382.749930] Lustre: DEBUG MARKER: oleg158-server.virtnet: executing set_default_debug -1 all [ 4390.696873] Lustre: DEBUG MARKER: oleg158-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 4395.880366] Lustre: DEBUG MARKER: oleg158-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 4402.120819] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 4402.531975] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 4405.620678] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:85 to 0x280000401:161) [ 4409.247447] Lustre: DEBUG MARKER: oleg158-server.virtnet: executing set_default_debug -1 all [ 4415.985901] Lustre: DEBUG MARKER: oleg158-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 4423.035327] Lustre: lustre-OST0000: new connection from lustre-MDT0001-mdtlov (cleaning up unused objects from 0x280000400:72 to 0x280000400:129) [ 4425.601386] Lustre: *** cfs_fail_loc=159, val=0*** [ 4441.937181] Lustre: lustre-MDT0000: Client 6b67fb19-7122-4ef5-8ed9-58e90dc6ae1f (at 192.168.201.58@tcp) reconnecting [ 4446.690629] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 4446.700519] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4446.718192] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 4448.645922] Lustre: server umount lustre-OST0000 complete [ 4453.858306] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4453.866180] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 4453.869505] Lustre: Skipped 1 previous similar message [ 4458.799772] Lustre: server umount lustre-MDT0000 complete [ 4458.977281] LustreError: 149881:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 4458.998708] LustreError: 149881:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 1 previous similar message [ 4463.459848] LustreError: 147320:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1787797470 with bad export cookie 9951052780431300490 [ 4463.480256] LustreError: MGC192.168.201.158@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 4474.043252] Lustre: server umount lustre-MDT0001 complete [ 4484.918471] Lustre: DEBUG MARKER: oleg158-server.virtnet: executing unload_modules_local [ 4488.075443] Key type lgssc unregistered [ 4488.564785] LNet: 151005:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 4488.584258] LNetError: 151005:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 4489.643475] LNet: Removed LNI 192.168.201.158@tcp [ 4490.796779] Key type .llcrypt unregistered [ 4490.800404] Key type ._llcrypt unregistered [ 4514.894527] Key type ._llcrypt registered [ 4514.898831] Key type .llcrypt registered [ 4514.985178] Lustre: DEBUG MARKER: oleg158-server.virtnet: executing load_modules_local [ 4516.037809] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 4516.135782] alg: No test for adler32 (adler32-zlib) [ 4517.328747] Lustre: Lustre: Build Version: 2.17.57_48_g018c4cb [ 4517.502977] LNet: Added LNI 192.168.201.158@tcp [8/256/0/180] [ 4519.159183] Key type lgssc registered [ 4520.323765] Lustre: Echo OBD driver; http://www.lustre.org/ [ 4527.427162] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 4527.441255] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 4528.866692] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 4533.552202] Lustre: DEBUG MARKER: oleg158-server.virtnet: executing set_default_debug -1 all [ 4538.352944] LustreError: 152421:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 4538.374583] LustreError: 152421:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 2 previous similar messages [ 4542.697712] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 4543.201673] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 4547.889787] Lustre: DEBUG MARKER: oleg158-server.virtnet: executing set_default_debug -1 all [ 4555.257735] Lustre: DEBUG MARKER: oleg158-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 4561.875851] Lustre: DEBUG MARKER: oleg158-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 4569.051822] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 4569.522598] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 4573.630253] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:170 to 0x280000401:193) [ 4576.387815] Lustre: DEBUG MARKER: oleg158-server.virtnet: executing set_default_debug -1 all [ 4584.884625] Lustre: DEBUG MARKER: oleg158-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 4588.987467] Lustre: lustre-OST0000: new connection from lustre-MDT0001-mdtlov (cleaning up unused objects from 0x280000400:72 to 0x280000400:161) [ 4611.103593] Lustre: lustre-MDT0001: evicting 380003ab-5592-4aef-b377-50981d6c570f (at 192.168.201.58@tcp) by administrative request [ 4611.115648] Lustre: Skipped 1 previous similar message [ 4645.355122] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4645.362830] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 4645.378094] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 4645.389343] LustreError: Skipped 1 previous similar message [ 4646.800979] Lustre: server umount lustre-OST0000 complete [ 4650.982252] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 4650.993898] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4651.005598] Lustre: Skipped 1 previous similar message [ 4651.027230] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 4651.038053] Lustre: Skipped 1 previous similar message [ 4655.586385] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4655.598290] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 4655.616975] Lustre: Skipped 1 previous similar message [ 4660.708055] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 4660.718217] Lustre: Skipped 1 previous similar message [ 4665.311490] Lustre: lustre-MDT0000 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 4665.500881] Lustre: server umount lustre-MDT0000 complete [ 4665.827559] LustreError: 155329:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 4665.848295] LustreError: 155329:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 1 previous similar message [ 4669.799178] LustreError: 152399:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1787797676 with bad export cookie 2402466301980807564 [ 4669.805953] LustreError: MGC192.168.201.158@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 4670.144894] Lustre: server umount lustre-MDT0001 complete [ 4682.119148] Lustre: DEBUG MARKER: oleg158-server.virtnet: executing unload_modules_local [ 4685.020804] Key type lgssc unregistered [ 4685.260846] LNet: 156492:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 4685.269557] LNetError: 156492:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 4685.291622] LNet: Removed LNI 192.168.201.158@tcp [ 4686.442229] Key type .llcrypt unregistered [ 4686.444870] Key type ._llcrypt unregistered [ 4773.712475] Key type ._llcrypt registered [ 4773.715320] Key type .llcrypt registered [ 4773.828797] Lustre: DEBUG MARKER: oleg158-server.virtnet: executing load_modules_local [ 4774.809329] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 4774.982540] alg: No test for adler32 (adler32-zlib) [ 4776.098881] Lustre: Lustre: Build Version: 2.17.57_48_g018c4cb [ 4776.419701] LNet: Added LNI 192.168.201.158@tcp [8/256/0/180] [ 4778.143210] Key type lgssc registered [ 4779.285415] Lustre: Echo OBD driver; http://www.lustre.org/ [ 4787.125760] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 4787.149654] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 4788.688838] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 4793.162933] Lustre: DEBUG MARKER: oleg158-server.virtnet: executing set_default_debug -1 all [ 4798.948234] LustreError: 158996:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 4798.963866] LustreError: 158996:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 2 previous similar messages [ 4802.351527] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 4802.668038] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 4806.974175] Lustre: DEBUG MARKER: oleg158-server.virtnet: executing set_default_debug -1 all [ 4814.206824] Lustre: DEBUG MARKER: oleg158-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 4818.917198] Lustre: DEBUG MARKER: oleg158-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 4826.020125] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 4826.380923] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 4827.388855] Lustre: lustre-OST0000: new connection from lustre-MDT0001-mdtlov (cleaning up unused objects from 0x280000400:72 to 0x280000400:193) [ 4832.908547] Lustre: DEBUG MARKER: oleg158-server.virtnet: executing set_default_debug -1 all [ 4833.524332] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:195 to 0x280000401:225) [ 4840.628322] Lustre: DEBUG MARKER: oleg158-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 4853.216032] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 4853.218973] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4853.225765] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 4854.240488] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 4854.250480] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 4854.259681] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4854.287675] Lustre: Skipped 1 previous similar message [ 4856.416706] Lustre: server umount lustre-OST0000 complete [ 4863.969331] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 4863.986409] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4864.011370] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 4866.568660] Lustre: server umount lustre-MDT0000 complete [ 4869.601865] LustreError: 158991:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 4869.619867] LustreError: 158991:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 1 previous similar message [ 4870.195860] LustreError: 158977:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1787797877 with bad export cookie 17789149572040531301 [ 4870.204468] LustreError: MGC192.168.201.158@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 4870.211260] LustreError: 158977:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 2 previous similar messages [ 4870.512121] Lustre: server umount lustre-MDT0001 complete [ 4881.829333] Lustre: DEBUG MARKER: oleg158-server.virtnet: executing unload_modules_local [ 4885.051457] Key type lgssc unregistered [ 4885.428862] LNet: 162539:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 4885.435781] LNetError: 162539:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 4885.454133] LNet: Removed LNI 192.168.201.158@tcp [ 4886.421841] Key type .llcrypt unregistered [ 4886.429341] Key type ._llcrypt unregistered [ 4911.917879] Key type ._llcrypt registered [ 4911.920365] Key type .llcrypt registered [ 4912.025480] Lustre: DEBUG MARKER: oleg158-server.virtnet: executing load_modules_local [ 4912.943498] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 4912.964591] alg: No test for adler32 (adler32-zlib) [ 4914.127272] Lustre: Lustre: Build Version: 2.17.57_48_g018c4cb [ 4914.437497] LNet: Added LNI 192.168.201.158@tcp [8/256/0/180] [ 4916.103218] Key type lgssc registered [ 4917.306935] Lustre: Echo OBD driver; http://www.lustre.org/ [ 4931.109410] Lustre: DEBUG MARKER: == conf-sanity test 100: check lshowmount lists MGS, MDT, OST and 0@lo ========================================================== 22:32:16 (1787797936) [ 4946.314516] Lustre: DEBUG MARKER: oleg158-server.virtnet: executing set_hostid [ 4956.968768] Lustre: DEBUG MARKER: oleg158-server.virtnet: executing load_modules_local [ 4990.854960] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 4990.885699] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 4992.237846] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 4992.278129] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 4992.366265] Lustre: lustre-MDT0000: new disk, initializing [ 4992.440890] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 4992.452254] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 4997.852165] Lustre: DEBUG MARKER: oleg158-server.virtnet: executing set_default_debug -1 all [ 5009.763516] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 5009.913822] Lustre: 166803:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 5009.947879] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [ 5009.952765] Lustre: Skipped 1 previous similar message [ 5010.098853] Lustre: lustre-MDT0001: new disk, initializing [ 5010.298881] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 5010.385163] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [ 5010.402590] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 5014.874236] Lustre: DEBUG MARKER: oleg158-server.virtnet: executing set_default_debug -1 all [ 5024.955859] Lustre: DEBUG MARKER: oleg158-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 5030.128303] Lustre: DEBUG MARKER: oleg158-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 5037.312782] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 5037.712861] Lustre: lustre-OST0000: new disk, initializing [ 5037.720224] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 5037.731928] Lustre: 168770:0:(osd_compat.c:1353:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 5037.887319] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 5043.184855] Lustre: DEBUG MARKER: oleg158-server.virtnet: executing set_default_debug -1 all [ 5046.350117] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [ 5046.359745] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 5046.532353] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 5051.624284] Lustre: DEBUG MARKER: oleg158-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 5062.113336] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 5062.130286] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5062.157182] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 5062.167870] Lustre: Skipped 1 previous similar message [ 5065.195533] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 5065.205847] Lustre: Skipped 1 previous similar message [ 5066.475415] Lustre: server umount lustre-OST0000 complete [ 5071.842596] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 5071.846300] LustreError: Skipped 1 previous similar message [ 5071.865119] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5071.895883] Lustre: Skipped 1 previous similar message [ 5071.906452] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 5075.426692] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 5075.438878] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5076.968495] Lustre: server umount lustre-MDT0000 complete [ 5080.463909] LustreError: 169632:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1787798087 with bad export cookie 9424470217040506205 [ 5080.478544] LustreError: MGC192.168.201.158@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 5080.485796] LustreError: 169632:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 5080.824648] Lustre: server umount lustre-MDT0001 complete [ 5092.847229] Lustre: DEBUG MARKER: oleg158-server.virtnet: executing unload_modules_local [ 5095.370425] Key type lgssc unregistered [ 5095.592655] LNet: 170678:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 5095.602523] LNetError: 170678:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 5095.617286] LNet: Removed LNI 192.168.201.158@tcp [ 5096.697864] Key type .llcrypt unregistered [ 5096.700294] Key type ._llcrypt unregistered [ 5123.252965] Key type ._llcrypt registered [ 5123.255304] Key type .llcrypt registered [ 5123.392440] Lustre: DEBUG MARKER: oleg158-server.virtnet: executing load_modules_local [ 5124.320118] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 5124.408157] alg: No test for adler32 (adler32-zlib) [ 5125.461750] Lustre: Lustre: Build Version: 2.17.57_48_g018c4cb [ 5125.791348] LNet: Added LNI 192.168.201.158@tcp [8/256/0/180] [ 5127.519613] Key type lgssc registered [ 5128.805279] Lustre: Echo OBD driver; http://www.lustre.org/ [ 5136.771381] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 5136.796528] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 5138.325930] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 5142.704840] Lustre: DEBUG MARKER: oleg158-server.virtnet: executing set_default_debug -1 all [ 5148.647582] LustreError: 172092:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 5148.674183] LustreError: 172092:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 2 previous similar messages [ 5150.522874] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 5151.039158] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 5154.812128] Lustre: DEBUG MARKER: oleg158-server.virtnet: executing set_default_debug -1 all [ 5161.638301] Lustre: DEBUG MARKER: oleg158-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 5166.453587] Lustre: DEBUG MARKER: oleg158-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 5172.450745] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 5172.822350] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 5178.511433] Lustre: DEBUG MARKER: oleg158-server.virtnet: executing set_default_debug -1 all [ 5183.103811] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:3 to 0x280000401:33) [ 5185.502867] Lustre: DEBUG MARKER: oleg158-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 5192.979690] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 5193.012863] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5193.031874] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 5193.040806] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5193.053459] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 5194.026962] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 5194.064419] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5194.076574] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 5194.087302] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5194.104202] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 5195.127811] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 5195.147755] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5195.155084] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 5195.160538] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5195.171611] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 5196.052575] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5196.927294] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5197.686883] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 5197.691942] Lustre: Skipped 2 previous similar messages [ 5197.708343] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5197.717450] Lustre: Skipped 2 previous similar messages [ 5197.722518] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 5197.728530] Lustre: Skipped 2 previous similar messages [ 5197.733028] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5197.741586] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 5197.752455] Lustre: Skipped 2 previous similar messages [ 5198.640065] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5199.677241] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5200.560786] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5201.577433] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5202.642638] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 5202.647299] Lustre: Skipped 4 previous similar messages [ 5202.658310] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5202.664297] Lustre: Skipped 4 previous similar messages [ 5202.666691] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 5202.668826] Lustre: Skipped 4 previous similar messages [ 5202.674908] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5202.693869] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 5202.703741] Lustre: Skipped 4 previous similar messages [ 5203.566247] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5204.428625] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5205.264387] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5206.247406] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5206.899540] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5207.751636] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5208.558699] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5209.387663] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5210.086075] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5211.251420] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 5211.260046] Lustre: Skipped 9 previous similar messages [ 5211.278944] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5211.291336] Lustre: Skipped 9 previous similar messages [ 5211.296749] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 5211.303600] Lustre: Skipped 9 previous similar messages [ 5211.307963] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5211.318583] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 5211.322699] Lustre: Skipped 9 previous similar messages [ 5212.057155] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5212.993972] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5213.793895] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5214.595974] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5215.293579] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5216.215609] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5217.196935] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5218.203420] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5218.917793] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5219.809228] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5220.536431] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5221.466037] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5222.376792] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5223.416438] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5224.277373] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5225.102341] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5225.938248] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5226.863229] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5227.580225] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 5227.587219] Lustre: Skipped 18 previous similar messages [ 5227.600393] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5227.614042] Lustre: Skipped 18 previous similar messages [ 5227.617764] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 5227.619869] Lustre: Skipped 18 previous similar messages [ 5227.624614] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5227.636471] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 5227.647081] Lustre: Skipped 18 previous similar messages [ 5228.446027] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5229.090558] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5229.928775] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5230.832612] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5231.571071] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5232.194415] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5233.033311] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5233.728333] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5234.580285] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5235.287617] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5236.170136] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5237.238134] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5238.152685] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5239.082472] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5239.980842] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5241.032821] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5241.875738] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5242.924174] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5243.706988] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5244.439492] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5245.301597] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5246.041698] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5246.866653] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5247.899053] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5248.713479] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5249.512263] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5250.405046] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5251.313670] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5252.022147] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5252.847609] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5253.688749] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5254.701803] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5255.362700] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5256.220908] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5256.870092] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5257.771047] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5258.573052] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5259.380712] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5260.101983] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 5260.106151] Lustre: Skipped 38 previous similar messages [ 5260.141773] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5260.153584] Lustre: Skipped 38 previous similar messages [ 5260.157227] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 5260.161481] Lustre: Skipped 38 previous similar messages [ 5260.165274] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5260.172768] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 5260.181564] Lustre: Skipped 38 previous similar messages [ 5260.884607] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5261.776408] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5262.564195] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5263.473240] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5264.477221] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5265.240924] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5266.112753] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5267.130845] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5268.063353] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5268.926974] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5269.783916] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5270.604401] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5271.357070] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5272.116853] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5272.942894] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5273.770263] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5274.567246] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5275.521269] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5276.246983] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5277.058480] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5277.938206] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5278.754355] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5279.566350] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5280.350226] LustreError: 172112:0:(osp_precreate.c:655:osp_precreate_send()) lustre-OST0000-osc-MDT0000: can't precreate: rc = -5 [ 5280.353550] LustreError: 172112:0:(osp_precreate.c:1359:osp_precreate_thread()) lustre-OST0000-osc-MDT0000: cannot precreate objects: rc = -5 [ 5280.367543] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5281.122779] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5282.048077] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5282.925125] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5283.748770] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5284.565067] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5285.385361] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5286.107512] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5286.887794] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5287.708753] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5288.528877] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5289.308532] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5290.134247] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5291.165747] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5292.067249] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5292.877390] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5293.717340] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5294.880290] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5295.829860] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5297.011780] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5297.907368] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5298.718447] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5299.503676] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5300.387579] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5301.068136] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5301.790726] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5302.567070] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5303.431823] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5304.343988] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5305.229318] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5306.012700] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5306.757755] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5307.500827] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5308.370227] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5309.266781] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5310.087403] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5310.991330] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5311.757191] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5312.734672] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5313.714316] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5314.666919] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5315.478772] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5316.233594] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5316.980181] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5317.899729] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5318.716740] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5319.517750] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5320.288983] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5321.146910] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5322.042881] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5322.986771] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5323.832237] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5324.706432] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 5324.710737] Lustre: Skipped 75 previous similar messages [ 5324.736386] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5324.743867] Lustre: Skipped 75 previous similar messages [ 5324.747081] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 5324.752393] Lustre: Skipped 75 previous similar messages [ 5324.754922] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5324.768943] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 5324.774304] Lustre: Skipped 75 previous similar messages [ 5325.667728] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5326.664471] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5327.411566] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5328.273226] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5328.993119] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5329.815362] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5330.546402] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5331.418338] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5332.137476] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5332.832180] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5333.638409] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5334.392930] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5335.217552] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5336.031578] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5336.877611] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5337.822892] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5338.570647] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5339.456803] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5340.211349] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5341.041489] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5341.787593] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5342.678761] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5343.444349] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5344.227966] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5345.126389] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5346.087353] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5346.846557] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5347.505395] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5348.244294] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5349.086285] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5349.949716] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5350.892627] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5351.785958] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5352.464355] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5353.261182] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5354.169195] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5354.918835] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5355.659513] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5356.302143] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5357.179585] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5357.755750] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5358.912458] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5359.833992] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5360.686034] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5361.654545] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5362.711832] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5363.797622] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5364.873519] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5365.854265] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5366.795206] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5367.642610] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5368.724336] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5369.846624] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5370.796106] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5371.920208] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5372.718385] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5373.656808] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5374.690907] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5375.563582] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5376.565085] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5377.530412] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5378.444648] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5379.395778] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5380.126375] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5380.924949] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5381.767910] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5382.657414] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5383.428382] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5384.254478] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5385.013218] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5385.711194] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5386.483557] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5387.214900] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5388.149332] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5388.853408] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5389.789288] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5390.549907] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5391.529948] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5392.391531] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5393.406196] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5394.138558] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5394.998739] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5395.784566] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5396.781420] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5397.747517] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5398.701655] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5399.470488] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5400.304609] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5401.005893] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5401.959979] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5402.884834] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5404.007887] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5404.904042] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5405.764469] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5406.516509] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5407.488170] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5408.452969] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5409.638377] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5410.569310] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5411.720934] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5412.909172] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5414.031764] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5415.190033] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5416.177520] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5417.178797] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5417.920716] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5418.742206] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5419.637236] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5420.731227] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5421.808732] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5422.925279] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5423.891558] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5424.891855] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5425.767362] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5426.809550] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5427.603402] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5428.562546] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5429.310690] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5430.515230] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5431.544441] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5432.429492] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5433.412859] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5434.420504] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5435.282237] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5436.139699] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5437.082370] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5437.985073] LustreError: 172112:0:(osp_precreate.c:655:osp_precreate_send()) lustre-OST0000-osc-MDT0000: can't precreate: rc = -5 [ 5437.998524] LustreError: 172112:0:(osp_precreate.c:1359:osp_precreate_thread()) lustre-OST0000-osc-MDT0000: cannot precreate objects: rc = -5 [ 5438.003198] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5439.040458] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5440.039426] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5440.752587] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5441.839639] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5442.699819] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5443.518880] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5444.387742] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5445.316274] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5446.286392] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5447.194662] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5448.136524] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5448.914583] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5449.871509] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5450.762755] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5451.712652] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5452.453724] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5453.286954] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 5453.294547] Lustre: Skipped 143 previous similar messages [ 5453.316790] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5453.334719] Lustre: Skipped 143 previous similar messages [ 5453.339143] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 5453.343391] Lustre: Skipped 143 previous similar messages [ 5453.350241] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5453.358377] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 5453.362721] Lustre: Skipped 143 previous similar messages [ 5454.116647] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5455.004053] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5455.912881] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5456.726093] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5457.632130] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5458.381361] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5459.172216] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5460.160283] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5461.131401] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5462.095241] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5462.917545] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5463.859791] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5464.823952] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5465.743045] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5466.617149] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5467.487718] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5468.578736] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5469.537306] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5470.478645] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5471.388986] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5472.394151] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5473.215486] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5474.165064] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5475.089580] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5475.959152] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5476.847858] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5477.850780] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5478.742834] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5479.656124] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5480.485545] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5481.369382] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5482.280539] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5483.380479] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5484.122488] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5484.966757] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5485.777805] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5486.611064] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5487.526655] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5488.512423] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5489.518740] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5490.374953] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5491.301960] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5492.199361] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5492.981969] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5493.954757] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5494.685476] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5495.451053] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5496.142382] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5496.929916] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5497.597521] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5498.482132] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5499.413828] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5500.412729] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5501.458590] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5502.314658] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5503.345516] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5504.190639] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5505.024152] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5505.739054] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5506.523791] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5507.389517] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5508.233433] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5509.099330] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5509.920765] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5510.574844] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5511.331122] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5512.108812] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5513.111031] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5513.866877] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5514.787775] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5515.558580] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5516.293687] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5517.181513] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5517.944956] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5518.584132] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5519.238969] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5520.035648] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5520.888616] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5521.532044] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5522.433918] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5523.215523] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5524.059644] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5524.756755] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5525.638904] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5526.447265] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5527.370059] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5528.213431] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5529.317600] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5530.123474] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5531.209597] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5532.191443] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5533.175627] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5533.976855] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5534.918543] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5535.665738] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5536.725919] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5537.616053] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5538.975776] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5540.051670] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5541.299542] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5542.384549] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5543.536080] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5544.557654] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5545.891564] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5547.061894] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5548.140361] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5549.213416] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5550.182099] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5551.318970] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5552.319223] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5553.136831] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5554.100773] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5554.658635] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5555.544662] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5556.502556] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5557.337392] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5558.124392] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5558.901567] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5559.523612] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5560.440645] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5561.749769] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5562.607243] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5563.535294] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5564.054160] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5564.856906] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5565.558150] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5566.285479] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5566.822254] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5567.481129] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5568.249242] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5569.040104] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5570.168991] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5571.241995] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5572.204570] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5573.270810] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5574.244283] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5575.277940] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5576.394912] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5577.675605] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5578.437896] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5579.778780] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5580.594334] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5581.459987] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5582.339615] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5583.545757] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5584.561802] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5585.379719] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5586.058983] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5586.923849] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5587.728599] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5588.673519] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5589.518264] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5590.466979] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5591.285361] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5592.015122] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5592.661095] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5593.476973] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5594.427730] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5595.391303] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5596.153087] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5597.229607] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5598.397350] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5599.525241] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5600.388671] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5601.493254] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5602.512489] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5603.643944] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5604.730431] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5605.488377] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5606.273476] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5607.009023] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5608.119396] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5609.274566] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5610.134940] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5611.085079] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5611.776694] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5612.652608] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5613.905159] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5614.955396] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5615.999381] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5617.077218] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5618.347667] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5619.455100] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5620.826454] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5621.677230] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5622.559348] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5623.351959] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5624.259828] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5625.037031] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5626.147129] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5627.076300] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5628.320445] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5629.214289] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5630.265528] LustreError: 172112:0:(osp_precreate.c:655:osp_precreate_send()) lustre-OST0000-osc-MDT0000: can't precreate: rc = -5 [ 5630.275278] LustreError: 172112:0:(osp_precreate.c:1359:osp_precreate_thread()) lustre-OST0000-osc-MDT0000: cannot precreate objects: rc = -5 [ 5630.278245] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5631.030838] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5632.305716] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5633.252337] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5634.876886] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5636.072979] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5636.873579] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5637.829134] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5638.562813] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5639.382138] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5640.091380] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5641.217807] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5642.150646] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5643.340638] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5644.137247] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5645.218673] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5645.988335] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5646.710543] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5647.606131] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5648.584826] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5649.483726] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5650.559695] LustreError: 172112:0:(osp_precreate.c:655:osp_precreate_send()) lustre-OST0000-osc-MDT0000: can't precreate: rc = -5 [ 5650.575454] LustreError: 172112:0:(osp_precreate.c:1359:osp_precreate_thread()) lustre-OST0000-osc-MDT0000: cannot precreate objects: rc = -5 [ 5650.580561] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5651.489285] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5652.383959] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5653.129401] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5653.945045] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5654.759232] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5655.598759] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5656.541040] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5657.400711] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5658.170300] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5659.002531] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5659.785929] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5660.713404] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5661.374622] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5662.189400] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5662.913733] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5663.709742] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5664.584376] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5665.563212] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5666.448493] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5667.405694] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5668.323904] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5669.292355] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5670.299607] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5671.297221] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5672.335376] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5673.477901] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5674.626547] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5675.737688] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5676.611639] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5677.450251] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5678.242206] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5679.168763] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5679.903460] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5680.779195] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5681.497363] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5682.279645] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5683.227284] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5684.073690] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5684.792158] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5685.837178] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5686.758114] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5687.685125] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5688.411240] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5689.498607] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5690.334564] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5691.126831] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5691.847590] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5692.638712] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5693.379608] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5694.208683] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5694.953736] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5695.854401] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5696.536655] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5697.288823] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5698.001335] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5698.799573] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5699.569918] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5700.387501] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5701.148717] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5701.857531] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5702.702609] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5703.497178] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5704.225240] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5705.052875] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5705.781893] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5706.470883] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5707.231740] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5708.058437] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5708.959941] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5709.813197] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 5709.818725] Lustre: Skipped 284 previous similar messages [ 5709.837172] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5709.843976] Lustre: Skipped 284 previous similar messages [ 5709.847075] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 5709.849837] Lustre: Skipped 284 previous similar messages [ 5709.853538] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5709.860293] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 5709.863020] Lustre: Skipped 284 previous similar messages [ 5710.659670] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5711.526076] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5712.384966] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5713.120218] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5713.867811] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5714.915966] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5715.966573] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5716.743420] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5717.551316] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5718.416382] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5719.292567] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5719.935792] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5720.656933] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5721.314539] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5722.012757] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5722.851408] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5723.659231] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5724.584606] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5725.303473] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5726.196548] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5726.977039] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5727.751289] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5728.393656] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5728.994645] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5729.787262] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5730.509980] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5731.359506] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5732.328892] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5733.073596] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5733.845497] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5734.759179] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5735.494027] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5736.272093] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5737.072159] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5737.900242] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5738.708538] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5739.604948] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5740.433726] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5741.368192] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5742.202598] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5743.009529] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5743.761588] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5744.604350] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5745.329067] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5746.132284] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5747.094440] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5747.820839] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5748.562958] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5749.282132] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5749.991995] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5750.809535] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5751.725224] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5752.393568] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5753.290376] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5753.948433] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5754.689464] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5755.585358] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5756.222906] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5756.810982] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5757.617345] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5758.355969] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5759.093446] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5759.789345] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5760.485980] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5761.112879] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5761.788620] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5762.420121] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5763.231854] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5763.903779] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5764.586699] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5765.277407] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5766.045070] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5766.983278] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5767.944807] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5768.910821] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5769.953402] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5770.639260] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5771.469521] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5772.226309] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5773.148573] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5773.923148] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5775.034311] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5776.314261] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5777.214954] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5778.281516] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5779.423350] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5780.406545] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5781.276237] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5782.115966] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5783.222400] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5784.132542] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5785.450070] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5786.340195] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5787.554322] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5788.408632] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5789.438513] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5790.280514] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5791.145487] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5792.003889] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5792.985841] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5793.736123] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5794.543038] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5795.098283] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5795.993924] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5796.755983] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5797.563611] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5798.322915] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5799.133975] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5800.036110] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5801.070196] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5801.810877] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5802.842486] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5803.760428] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5804.593056] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5805.577382] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5806.364199] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5807.476373] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5808.356302] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5809.174229] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5809.898304] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5810.693882] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5811.830364] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5812.557802] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5813.219230] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5813.843781] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5814.593667] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5815.250930] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5815.935214] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5816.556518] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5817.285367] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5818.147873] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5819.004866] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5819.919794] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5820.770556] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5821.416033] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5822.345593] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5823.187427] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5823.960701] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5825.118250] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5826.268069] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5826.985462] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5827.689672] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5828.375848] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5829.379554] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5830.498907] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5831.488218] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5832.601682] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5833.579937] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5834.668629] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5835.597836] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5836.555196] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5837.722670] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5839.001900] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5840.009279] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5841.131234] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5842.089389] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5843.277594] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5844.245567] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5845.249656] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5846.226620] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5847.344218] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5848.327262] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5849.226463] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5850.304866] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5851.248471] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5852.217149] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5853.008675] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5853.958839] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5854.839184] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5855.817936] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5856.805640] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5857.632278] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5858.566558] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5859.631355] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5860.426414] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5861.578524] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5862.417771] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5863.292902] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5864.208094] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5865.246392] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5866.203357] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5867.179990] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5867.937888] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5868.911169] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5869.644572] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5870.473290] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5871.237094] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5871.979352] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5872.712569] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5873.522748] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5874.262767] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5875.049872] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5875.758721] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5876.436811] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5877.273305] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5877.985386] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5878.648027] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5879.539962] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5880.435192] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5881.235961] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5882.091080] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5882.895819] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5883.727647] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5884.659733] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5885.661628] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5886.881771] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5887.834955] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5888.688064] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5889.386763] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5890.209254] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5890.989325] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5892.021577] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5892.780199] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5893.602599] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5894.328135] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5895.244245] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5896.069492] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5896.849165] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5897.628434] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5898.592497] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5899.557638] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5900.408594] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5901.304274] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5902.458496] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5903.315745] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5904.451735] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5905.599220] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5906.636355] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5907.810693] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5908.912820] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5910.115381] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5910.798687] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5911.548892] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5912.490049] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5913.708047] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5914.878469] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5916.227481] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5918.032836] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5919.226134] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5920.122495] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5921.211189] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5921.960610] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5923.033790] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5924.046529] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5925.454043] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5926.846858] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5928.684182] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5930.215974] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5931.530177] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5932.607821] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5934.224814] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5935.470126] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5936.598117] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5937.482712] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5938.300060] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5939.318936] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5940.369386] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5941.315258] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5942.649749] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5943.419177] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5944.286578] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5945.140342] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5946.325610] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5947.352363] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5948.255416] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5949.119522] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5950.032673] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5950.905227] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5951.910276] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5952.865204] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5953.686672] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5954.563099] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5955.284268] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5956.138852] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5956.932529] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5957.680201] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5958.744494] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5959.849582] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5960.762300] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5961.613541] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5962.227792] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5963.080893] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5963.854875] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5964.753188] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5965.834310] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5966.743865] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5967.786600] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5968.878767] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5969.817756] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5970.599527] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5971.358224] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5972.292207] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5973.253442] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5974.128196] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5975.227790] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5976.100648] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5976.973078] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5977.627208] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5978.599255] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5979.450950] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5980.197363] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5981.093730] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5982.016047] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5982.991756] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5983.738444] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5984.650403] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5985.299543] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5986.188081] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5987.191407] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5988.047332] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5988.929200] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5989.788776] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5990.535933] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5991.334937] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5992.134478] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5993.067568] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5993.753496] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5994.542975] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5995.333048] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5996.218876] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5996.914875] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5997.768346] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5998.585584] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5999.434398] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6000.424443] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6001.144892] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6001.947100] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6002.645422] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6003.642855] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6004.442268] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6005.408096] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6006.121922] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6007.072144] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6007.797919] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6008.827781] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6010.060160] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6010.907800] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6011.865374] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6012.604498] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6013.315138] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6014.073333] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6014.929764] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6015.687893] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6016.623457] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6017.324434] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6018.081314] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6018.919212] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6019.729498] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6020.461874] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6021.283362] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6022.214432] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6022.979214] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6023.654465] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6024.573145] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6025.377488] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6026.335630] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6027.286026] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6028.445686] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6029.683888] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6030.879849] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6031.973759] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6032.864792] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6033.763969] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6034.549764] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6035.423947] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6036.295971] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6037.526871] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6038.295214] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6039.202681] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6040.153168] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6041.233853] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6041.978870] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6042.996101] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6043.980992] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6045.113835] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6046.288765] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6047.371396] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6048.402388] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6049.509148] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6050.278780] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6051.025905] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6051.902352] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6052.855420] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6053.525572] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6054.278782] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6055.046171] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6055.823203] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6056.587712] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6057.174208] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6057.954383] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6058.599816] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6059.394889] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6060.325584] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6061.363938] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6062.453804] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6063.493417] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6064.889763] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6065.931870] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6066.814222] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6067.917766] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6068.841705] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6069.490376] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6070.260939] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6071.001390] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6071.813746] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6072.547355] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6073.456139] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6074.347857] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6075.088166] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6076.129191] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6076.888600] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6078.011282] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6078.890541] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6079.895164] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6080.744450] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6081.705426] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6446.331242] LustreError: 173912:0:(ofd_dev.c:1780:ofd_create_hdl()) lustre-OST0000: unable to precreate: rc = -28 [ 6446.343983] LustreError: 172112:0:(osp_precreate.c:655:osp_precreate_send()) lustre-OST0000-osc-MDT0000: can't precreate: rc = -28 [ 6890.466303] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 6890.502955] Lustre: Skipped 416 previous similar messages [ 6890.514146] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 6893.521895] Lustre: server umount lustre-OST0000 complete [ 6897.120884] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 6897.127775] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 6897.130936] Lustre: Skipped 1 previous similar message [ 6900.705211] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 6900.718260] Lustre: Skipped 1 previous similar message [ 6903.393307] Lustre: server umount lustre-MDT0000 complete [ 6905.830492] LustreError: 172821:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 6905.848425] LustreError: 172821:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 1 previous similar message [ 6907.334734] LustreError: 172071:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1787799914 with bad export cookie 9854593846144068631 [ 6907.339624] LustreError: MGC192.168.201.158@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 6907.341903] LustreError: 172071:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 6907.851316] Lustre: server umount lustre-MDT0001 complete [ 6920.350874] Lustre: DEBUG MARKER: oleg158-server.virtnet: executing unload_modules_local [ 6922.959048] Key type lgssc unregistered [ 6923.328216] LNet: 227333:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 6923.350263] LNetError: 227333:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 6924.391793] LNet: Removed LNI 192.168.201.158@tcp [ 6925.580260] Key type .llcrypt unregistered [ 6925.584197] Key type ._llcrypt unregistered [ 6953.143346] Key type ._llcrypt registered [ 6953.145854] Key type .llcrypt registered [ 6953.253242] Lustre: DEBUG MARKER: oleg158-server.virtnet: executing load_modules_local [ 6954.477864] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 6954.719911] alg: No test for adler32 (adler32-zlib) [ 6955.852265] Lustre: Lustre: Build Version: 2.17.57_48_g018c4cb [ 6956.076552] LNet: Added LNI 192.168.201.158@tcp [8/256/0/180] [ 6957.751162] Key type lgssc registered [ 6958.764995] Lustre: Echo OBD driver; http://www.lustre.org/ [ 6965.606979] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 6965.632253] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 6967.257374] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 6971.780030] Lustre: DEBUG MARKER: oleg158-server.virtnet: executing set_default_debug -1 all [ 6977.505410] LustreError: 228750:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 6977.517817] LustreError: 228750:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 2 previous similar messages [ 6980.500085] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 6980.794481] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 6985.126442] Lustre: DEBUG MARKER: oleg158-server.virtnet: executing set_default_debug -1 all [ 6992.502669] Lustre: DEBUG MARKER: oleg158-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 6999.408242] Lustre: DEBUG MARKER: oleg158-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 7006.357548] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 7006.819561] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 7013.002176] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:49665 to 0x280000401:49696) [ 7014.453121] Lustre: DEBUG MARKER: oleg158-server.virtnet: executing set_default_debug -1 all [ 7024.137835] Lustre: DEBUG MARKER: oleg158-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 7031.220298] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 7033.365103] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 7033.376513] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 7033.383506] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7033.392620] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 7033.395407] LustreError: 228770:0:(osp_precreate.c:1268:osp_precreate_thread()) cfs_fail_timeout id 2107 sleeping for 20000ms [ 7035.874154] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 7035.875213] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 7035.882366] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 7035.890028] Lustre: Skipped 1 previous similar message [ 7037.727568] Lustre: lustre-OST0000: Not available for connect from 192.168.201.58@tcp (stopping) [ 7037.739903] Lustre: Skipped 1 previous similar message [ 7041.007487] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 7041.018216] Lustre: Skipped 1 previous similar message [ 7041.966932] Lustre: server umount lustre-OST0000 complete [ 7053.415463] LustreError: 228770:0:(osp_precreate.c:1268:osp_precreate_thread()) cfs_fail_timeout id 2107 awake [ 7077.480587] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 7077.630800] LustreError: 231959:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-OST0000: not available for connect from 192.168.201.58@tcp (no target). If you are running an HA pair check that the target is mounted on the other server. [ 7077.990657] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 7084.450844] Lustre: DEBUG MARKER: oleg158-server.virtnet: executing set_default_debug -1 all [ 7091.305118] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7091.314821] LustreError: lustre-OST0000-osc-MDT0001: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7091.315233] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 7091.890674] Lustre: DEBUG MARKER: oleg158-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 7093.752218] Lustre: DEBUG MARKER: osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid in FULL state after 0 sec [ 7098.892959] Lustre: DEBUG MARKER: oleg158-client.virtnet: executing wait_import_state (FULL|IDLE) osc.lustre-OST0000-osc-ffff9979d8765800.ost_server_uuid 50 [ 7100.509558] Lustre: DEBUG MARKER: osc.lustre-OST0000-osc-ffff9979d8765800.ost_server_uuid in FULL state after 0 sec [ 7107.046598] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 7107.046868] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 7107.056450] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 7110.623421] Lustre: server umount lustre-OST0000 complete [ 7114.209044] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 7114.216633] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 7114.231289] Lustre: Skipped 1 previous similar message [ 7114.235407] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 7114.238206] Lustre: Skipped 1 previous similar message [ 7120.311143] Lustre: server umount lustre-MDT0000 complete [ 7122.402934] LustreError: 229804:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 7122.413643] LustreError: 229804:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 1 previous similar message [ 7123.550145] LustreError: 228730:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1787800130 with bad export cookie 15715583314287342318 [ 7123.563110] LustreError: MGC192.168.201.158@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 7123.565609] LustreError: 228730:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 2 previous similar messages [ 7123.880723] Lustre: server umount lustre-MDT0001 complete [ 7136.050485] Lustre: DEBUG MARKER: oleg158-server.virtnet: executing unload_modules_local [ 7138.678558] Key type lgssc unregistered [ 7138.935700] LNet: 233964:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 7138.946246] LNetError: 233964:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 7138.964122] LNet: Removed LNI 192.168.201.158@tcp [ 7139.800850] Key type .llcrypt unregistered [ 7139.806099] Key type ._llcrypt unregistered [ 7173.478787] Key type ._llcrypt registered [ 7173.481209] Key type .llcrypt registered [ 7173.621189] Lustre: DEBUG MARKER: oleg158-server.virtnet: executing load_modules_local [ 7174.878278] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 7175.004220] alg: No test for adler32 (adler32-zlib) [ 7176.089876] Lustre: Lustre: Build Version: 2.17.57_48_g018c4cb [ 7176.386103] LNet: Added LNI 192.168.201.158@tcp [8/256/0/180] [ 7178.063175] Key type lgssc registered [ 7179.359515] Lustre: Echo OBD driver; http://www.lustre.org/ [ 7187.988628] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 7187.995815] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 7189.545312] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 7193.317721] Lustre: DEBUG MARKER: oleg158-server.virtnet: executing set_default_debug -1 all [ 7199.724675] LustreError: 235711:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 7199.753500] LustreError: 235711:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 2 previous similar messages [ 7202.929686] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 7203.435589] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 7207.822589] Lustre: DEBUG MARKER: oleg158-server.virtnet: executing set_default_debug -1 all [ 7210.373397] Lustre: Modifying parameter general.debug_raw_pointers=Y in log params [ 7218.062208] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 7218.419802] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 7224.718583] Lustre: DEBUG MARKER: oleg158-server.virtnet: executing set_default_debug -1 all [ 7233.063133] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 7233.236219] Lustre: lustre-OST0001: new disk, initializing [ 7233.239553] Lustre: srv-lustre-OST0001: No data found on store. Initialize space. [ 7233.244429] Lustre: 238015:0:(osd_compat.c:1353:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 7233.325595] Lustre: lustre-OST0001: Imperative Recovery not enabled, recovery window 60-180 [ 7234.359116] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:49698 to 0x280000401:49728) [ 7238.707791] Lustre: DEBUG MARKER: oleg158-server.virtnet: executing set_default_debug -1 all [ 7242.891465] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x00000002c0000400-0x0000000300000400]:1:ost [ 7242.906666] Lustre: cli-lustre-OST0001-super: Allocated super-sequence [0x00000002c0000400-0x0000000300000400]:1:ost] [ 7243.009251] Lustre: lustre-OST0001-osc-MDT0000: update sequence from 0x100010000 to 0x2c0000401 [ 7249.740724] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 7257.050604] Lustre: Setting parameter general.lod.*.mdt_hash=crush in log params [ 7273.965219] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 7273.974834] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 7273.985121] Lustre: Skipped 2 previous similar messages [ 7274.001541] Lustre: Skipped 3 previous similar messages [ 7279.076515] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 7279.106670] Lustre: Skipped 3 previous similar messages [ 7279.325891] Lustre: server umount lustre-MDT0000 complete [ 7282.951726] LustreError: 236445:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1787800289 with bad export cookie 7299988097802910704 [ 7282.956572] LustreError: MGC192.168.201.158@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 7282.963755] LustreError: 236445:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 1 previous similar message [ 7283.457972] Lustre: server umount lustre-MDT0001 complete [ 7298.142692] Lustre: server umount lustre-OST0000 complete [ 7300.385117] Lustre: 234972:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1787800291/real 1787800291] req@ffffa03a3a8f1c00 x1874644364780288/t0(0) o400->lustre-MDT0001-lwp-OST0001@0@lo:12/10 lens 224/224 e 0 to 1 dl 1787800307 ref 1 fl Rpc:XNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 [ 7300.420819] Lustre: lustre-MDT0001-lwp-OST0001: Connection to lustre-MDT0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 7300.437162] Lustre: Skipped 1 previous similar message [ 7301.915771] Lustre: server umount lustre-OST0001 complete [ 7322.690433] Lustre: DEBUG MARKER: oleg158-server.virtnet: executing load_modules_local [ 7332.592748] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 7333.007125] LustreError: 241125:0:(ldlm_lib.c:1190:target_handle_connect()) mylustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 7333.070324] Lustre: mylustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 7337.524146] Lustre: DEBUG MARKER: oleg158-server.virtnet: executing set_default_debug -1 all [ 7338.464309] LustreError: 241126:0:(ldlm_lib.c:1190:target_handle_connect()) mylustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 7343.596255] LustreError: 241125:0:(ldlm_lib.c:1190:target_handle_connect()) mylustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 7346.367676] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 7346.655646] Lustre: mylustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 7351.057611] Lustre: DEBUG MARKER: oleg158-server.virtnet: executing set_default_debug -1 all [ 7353.401282] Lustre: 242267:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify general/debug_raw_pointers=Y (mode = 0) failed: rc = -17 [ 7359.279331] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 7365.261823] Lustre: DEBUG MARKER: oleg158-server.virtnet: executing set_default_debug -1 all [ 7371.952769] LustreError: 242621:0:(ldlm_lib.c:1190:target_handle_connect()) mylustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 7373.707701] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 7374.009627] Lustre: mylustre-OST0001: Imperative Recovery not enabled, recovery window 60-180 [ 7374.026824] Lustre: Skipped 1 previous similar message [ 7378.139073] Lustre: mylustre-OST0001: new connection from mylustre-MDT0000-mdtlov (cleaning up unused objects from 0x2c0000401:3 to 0x2c0000401:33) [ 7378.143171] Lustre: mylustre-OST0000: new connection from mylustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:49730 to 0x280000401:49760) [ 7380.369973] Lustre: DEBUG MARKER: oleg158-server.virtnet: executing set_default_debug -1 all [ 7387.893884] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 7391.404306] Lustre: 244138:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify general/lod.*.mdt_hash=crush (mode = 0) failed: rc = -17 [ 7402.980995] LustreError: mylustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 7402.987562] Lustre: mylustre-MDT0000-osp-MDT0001: Connection to mylustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 7403.010482] Lustre: mylustre-MDT0000: Not available for connect from 0@lo (stopping) [ 7407.836055] Lustre: server umount mylustre-MDT0000 complete [ 7409.129615] LustreError: 241125:0:(ldlm_lib.c:1190:target_handle_connect()) mylustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 7409.148842] LustreError: 241125:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 3 previous similar messages [ 7411.672312] LustreError: 241102:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1787800418 with bad export cookie 7299988097802913511 [ 7411.683646] LustreError: 241102:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 2 previous similar messages [ 7411.684184] LustreError: MGC192.168.201.158@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 7412.098740] Lustre: server umount mylustre-MDT0001 complete [ 7427.031403] Lustre: server umount mylustre-OST0000 complete [ 7441.012211] Lustre: server umount mylustre-OST0001 complete [ 7464.727300] Lustre: DEBUG MARKER: oleg158-server.virtnet: executing load_modules_local [ 7474.605776] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 7474.986548] LustreError: 246447:0:(ldlm_lib.c:1190:target_handle_connect()) tfs-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 7475.062491] Lustre: tfs-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 7479.371252] Lustre: DEBUG MARKER: oleg158-server.virtnet: executing set_default_debug -1 all [ 7488.024376] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 7492.765715] Lustre: DEBUG MARKER: oleg158-server.virtnet: executing set_default_debug -1 all [ 7495.902921] Lustre: 247586:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify general/debug_raw_pointers=Y (mode = 0) failed: rc = -17 [ 7503.574961] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 7511.075837] Lustre: DEBUG MARKER: oleg158-server.virtnet: executing set_default_debug -1 all [ 7513.065141] LustreError: 247943:0:(ldlm_lib.c:1190:target_handle_connect()) tfs-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 7513.093045] LustreError: 247943:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 4 previous similar messages [ 7519.245437] Lustre: tfs-OST0000: new connection from tfs-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:49730 to 0x280000401:49792) [ 7519.425436] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 7520.680812] Lustre: tfs-OST0001: new connection from tfs-MDT0000-mdtlov (cleaning up unused objects from 0x2c0000401:35 to 0x2c0000401:65) [ 7526.214550] Lustre: DEBUG MARKER: oleg158-server.virtnet: executing set_default_debug -1 all [ 7534.237795] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 7537.983741] Lustre: 249459:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify general/lod.*.mdt_hash=crush (mode = 0) failed: rc = -17 [ 7546.341186] Lustre: tfs-MDT0000-osp-MDT0001: Connection to tfs-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 7546.352917] Lustre: tfs-MDT0000: Not available for connect from 0@lo (stopping) [ 7546.371662] Lustre: Skipped 6 previous similar messages [ 7546.381730] Lustre: Skipped 6 previous similar messages [ 7547.205691] Lustre: server umount tfs-MDT0000 complete [ 7551.348149] LustreError: 246422:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1787800558 with bad export cookie 7299988097802915401 [ 7551.351657] LustreError: MGC192.168.201.158@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 7551.375611] LustreError: 246422:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 7551.461365] Lustre: tfs-MDT0001-lwp-OST0001: Connection to tfs-MDT0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 7551.468493] Lustre: tfs-MDT0001: Not available for connect from 0@lo (stopping) [ 7551.477608] Lustre: Skipped 1 previous similar message [ 7551.492182] Lustre: Skipped 2 previous similar messages [ 7551.677724] Lustre: server umount tfs-MDT0001 complete [ 7558.071859] Lustre: server umount tfs-OST0000 complete [ 7562.214500] Lustre: server umount tfs-OST0001 complete [ 7586.588790] Lustre: DEBUG MARKER: oleg158-server.virtnet: executing load_modules_local [ 7598.001031] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 7598.400257] LustreError: 251709:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 7598.425075] LustreError: 251709:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 4 previous similar messages [ 7598.484384] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 7598.494344] Lustre: Skipped 3 previous similar messages [ 7603.435914] Lustre: DEBUG MARKER: oleg158-server.virtnet: executing set_default_debug -1 all [ 7611.613417] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 7616.679361] Lustre: DEBUG MARKER: oleg158-server.virtnet: executing set_default_debug -1 all [ 7619.318053] Lustre: 252852:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify general/debug_raw_pointers=Y (mode = 0) failed: rc = -17 [ 7625.661947] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 7631.957225] Lustre: DEBUG MARKER: oleg158-server.virtnet: executing set_default_debug -1 all [ 7641.260912] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 7643.592576] Lustre: lustre-OST0001: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x2c0000401:35 to 0x2c0000401:97) [ 7643.597435] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:49730 to 0x280000401:49824) [ 7647.435637] Lustre: DEBUG MARKER: oleg158-server.virtnet: executing set_default_debug -1 all [ 7655.507276] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 7659.210500] Lustre: 254725:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify general/lod.*.mdt_hash=crush (mode = 0) failed: rc = -17 [ 7669.642515] Lustre: DEBUG MARKER: == conf-sanity test 104a: Make sure user defined options are reflected in mount ========================================================== 23:17:53 (1787800673) [ 7688.672939] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 7688.689190] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 7688.707612] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 7692.257409] Lustre: server umount lustre-MDT0000 complete [ 7696.067578] LustreError: 251684:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1787800703 with bad export cookie 7299988097802917067 [ 7696.076320] LustreError: MGC192.168.201.158@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 7702.628820] Lustre: server umount lustre-MDT0001 complete [ 7712.614061] Lustre: server umount lustre-OST0000 complete [ 7722.759060] Lustre: server umount lustre-OST0001 complete [ 7729.412935] Lustre: DEBUG MARKER: oleg158-server.virtnet: executing set_hostid [ 7741.132710] Lustre: DEBUG MARKER: oleg158-server.virtnet: executing load_modules_local [ 7783.501714] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 7783.903467] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 7783.933597] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 7784.067274] Lustre: lustre-MDT0000: new disk, initializing [ 7784.199193] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 188-180 [ 7784.205480] Lustre: Skipped 3 previous similar messages [ 7784.226052] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 7789.065826] Lustre: DEBUG MARKER: oleg158-server.virtnet: executing set_default_debug -1 all [ 7799.824747] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 7800.032862] Lustre: 257913:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 7800.141373] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [ 7800.145728] Lustre: Skipped 1 previous similar message [ 7800.214481] Lustre: lustre-MDT0001: new disk, initializing [ 7800.330172] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [ 7800.346281] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 7805.475859] Lustre: DEBUG MARKER: oleg158-server.virtnet: executing set_default_debug -1 all [ 7815.595206] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 7815.861235] Lustre: lustre-OST0000: new disk, initializing [ 7815.865438] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 7815.871365] Lustre: 259493:0:(osd_compat.c:1353:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 7817.172803] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [ 7817.182241] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 7817.308892] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 7822.174531] Lustre: DEBUG MARKER: oleg158-server.virtnet: executing set_default_debug -1 all [ 7832.733224] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 7832.976702] Lustre: lustre-OST0001: new disk, initializing [ 7832.980879] Lustre: srv-lustre-OST0001: No data found on store. Initialize space. [ 7832.989071] Lustre: 260362:0:(osd_compat.c:1353:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 7834.377599] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x00000002c0000400-0x0000000300000400]:1:ost [ 7834.398741] Lustre: cli-lustre-OST0001-super: Allocated super-sequence [0x00000002c0000400-0x0000000300000400]:1:ost] [ 7834.548181] Lustre: lustre-OST0001-osc-MDT0000: update sequence from 0x100010000 to 0x2c0000401 [ 7840.826100] Lustre: DEBUG MARKER: oleg158-server.virtnet: executing set_default_debug -1 all [ 7855.867388] Lustre: DEBUG MARKER: == conf-sanity test 104b: Mount uses last flock argument ========================================================== 23:21:00 (1787800860) [ 7879.302149] Lustre: DEBUG MARKER: == conf-sanity test 105: check file creation for ro and rw bind mnt pt ========================================================== 23:21:23 (1787800883) [ 7896.038196] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 7896.047956] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 7896.052024] Lustre: Skipped 6 previous similar messages [ 7896.062146] Lustre: Skipped 9 previous similar messages [ 7909.343180] Lustre: lustre-OST0000 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 7909.575828] Lustre: server umount lustre-OST0000 complete [ 7911.400014] LustreError: 261367:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-OST0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 7911.417929] LustreError: 261367:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 12 previous similar messages [ 7920.651769] Lustre: server umount lustre-MDT0000 complete [ 7925.114472] LustreError: 258715:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1787800932 with bad export cookie 7299988097802918621 [ 7925.125834] LustreError: MGC192.168.201.158@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 7925.144609] LustreError: 258715:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 3 previous similar messages [ 7930.784946] Lustre: lustre-MDT0001: Not available for connect from 0@lo (stopping) [ 7930.791153] Lustre: Skipped 8 previous similar messages [ 7931.801367] Lustre: server umount lustre-MDT0001 complete [ 7944.997764] Lustre: DEBUG MARKER: oleg158-server.virtnet: executing unload_modules_local [ 8017.682593] Lustre: server umount lustre-OST0001 complete [ 8025.080826] Lustre: DEBUG MARKER: oleg158-server.virtnet: executing set_hostid [ 8042.117941] Lustre: DEBUG MARKER: oleg158-server.virtnet: executing load_modules_local [ 8079.618046] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 8079.937499] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 8079.966597] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 8080.044241] Lustre: lustre-MDT0000: new disk, initializing [ 8080.111792] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 8080.114247] Lustre: Skipped 3 previous similar messages [ 8080.131513] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 8084.398823] Lustre: DEBUG MARKER: oleg158-server.virtnet: executing set_default_debug -1 all [ 8095.717681] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 8095.820642] Lustre: 265255:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 8096.188091] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [ 8096.198224] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 8100.285616] Lustre: DEBUG MARKER: oleg158-server.virtnet: executing set_default_debug -1 all [ 8110.661503] Lustre: DEBUG MARKER: oleg158-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 8117.768845] Lustre: DEBUG MARKER: oleg158-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 8125.532688] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 8125.909871] Lustre: lustre-OST0000: new disk, initializing [ 8125.911855] Lustre: Skipped 1 previous similar message [ 8125.917787] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 8125.928320] Lustre: Skipped 2 previous similar messages [ 8125.934617] Lustre: 267222:0:(osd_compat.c:1353:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 8127.805458] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 8127.950048] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 8133.757709] Lustre: DEBUG MARKER: oleg158-server.virtnet: executing set_default_debug -1 all [ 8146.031449] Lustre: DEBUG MARKER: oleg158-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 8158.690706] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 8158.694193] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 8158.705200] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 8158.719064] Lustre: Skipped 5 previous similar messages [ 8160.573492] Lustre: server umount lustre-OST0000 complete [ 8167.909892] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 8171.124333] Lustre: server umount lustre-MDT0000 complete [ 8174.048336] LustreError: 265265:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 8174.087039] LustreError: 265265:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 8 previous similar messages [ 8174.857688] LustreError: 265245:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1787801181 with bad export cookie 7299988097802920091 [ 8174.868870] LustreError: MGC192.168.201.158@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 8174.873495] LustreError: 265245:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 8175.384697] Lustre: server umount lustre-MDT0001 complete [ 8187.694601] Lustre: DEBUG MARKER: oleg158-server.virtnet: executing unload_modules_local [ 8190.442857] Key type lgssc unregistered [ 8190.828413] LNet: 268987:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 8190.844596] LNetError: 268987:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 8190.861622] LNet: Removed LNI 192.168.201.158@tcp [ 8191.666179] Key type .llcrypt unregistered [ 8191.668152] Key type ._llcrypt unregistered [ 8222.866685] Key type ._llcrypt registered [ 8222.869984] Key type .llcrypt registered [ 8222.975712] Lustre: DEBUG MARKER: oleg158-server.virtnet: executing load_modules_local [ 8224.069394] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 8224.091217] alg: No test for adler32 (adler32-zlib) [ 8225.126590] Lustre: Lustre: Build Version: 2.17.57_48_g018c4cb [ 8225.384047] LNet: Added LNI 192.168.201.158@tcp [8/256/0/180] [ 8227.087146] Key type lgssc registered [ 8228.156951] Lustre: Echo OBD driver; http://www.lustre.org/ [ 8235.546495] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 8235.566415] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 8236.905849] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 8242.644858] Lustre: DEBUG MARKER: oleg158-server.virtnet: executing set_default_debug -1 all [ 8246.254898] LustreError: 270455:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 8246.275255] LustreError: 270455:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 2 previous similar messages [ 8251.361497] LustreError: 270454:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 8251.783427] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 8252.471185] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 8256.917564] Lustre: DEBUG MARKER: oleg158-server.virtnet: executing set_default_debug -1 all [ 8264.231989] Lustre: DEBUG MARKER: oleg158-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 8269.771272] Lustre: DEBUG MARKER: oleg158-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 8276.180044] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 8276.654601] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 8281.776945] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:3 to 0x280000401:33) [ 8283.131469] Lustre: DEBUG MARKER: oleg158-server.virtnet: executing set_default_debug -1 all [ 8291.069659] Lustre: DEBUG MARKER: oleg158-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 8292.050937] Lustre: Setting parameter lustre-OST0000.ost.unknown_param=50 in log lustre-OST0000 [ 8297.444691] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 8297.453049] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 8297.457128] Lustre: Skipped 1 previous similar message [ 8297.473911] Lustre: Skipped 1 previous similar message [ 8300.170587] Lustre: server umount lustre-OST0000 complete [ 8303.588899] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 8303.594591] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 8303.611754] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 8307.680600] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 8307.682527] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 8307.721056] Lustre: Skipped 1 previous similar message [ 8309.854559] Lustre: server umount lustre-MDT0000 complete [ 8312.810944] LustreError: 270450:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 8312.832276] LustreError: 270450:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 1 previous similar message [ 8313.490955] LustreError: 270436:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1787801320 with bad export cookie 12162683428673034386 [ 8313.492975] LustreError: MGC192.168.201.158@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 8313.503579] LustreError: 270436:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 1 previous similar message [ 8313.918954] Lustre: server umount lustre-MDT0001 complete [ 8328.391424] Lustre: DEBUG MARKER: oleg158-server.virtnet: executing unload_modules_local [ 8331.724512] Key type lgssc unregistered [ 8332.160587] LNet: 274044:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 8332.171825] LNetError: 274044:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 8332.184736] LNet: Removed LNI 192.168.201.158@tcp [ 8333.070326] Key type .llcrypt unregistered [ 8333.071612] Key type ._llcrypt unregistered [ 8351.301656] Key type ._llcrypt registered [ 8351.302967] Key type .llcrypt registered [ 8351.361608] Lustre: DEBUG MARKER: oleg158-server.virtnet: executing load_modules_local [ 8352.153983] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 8352.182108] alg: No test for adler32 (adler32-zlib) [ 8353.150426] Lustre: Lustre: Build Version: 2.17.57_48_g018c4cb [ 8353.281398] LNet: Added LNI 192.168.201.158@tcp [8/256/0/180] [ 8354.895737] Key type lgssc registered [ 8355.987387] Lustre: Echo OBD driver; http://www.lustre.org/ [ 8364.738795] Lustre: lustre-OST0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 8364.759742] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 8380.319988] LustreError: Failed to get MGS log lustre-sptlrpc and no local copy. [ 8385.439617] LustreError: 275238:0:(mgc_request.c:233:do_config_log_add()) MGC192.168.201.158@tcp: failed processing log, type 4: rc = -110 [ 8412.063201] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 8418.309963] Lustre: DEBUG MARKER: oleg158-server.virtnet: executing set_default_debug -1 all [ 8426.441963] Lustre: DEBUG MARKER: oleg158-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 8428.769651] Lustre: server umount lustre-OST0000 complete [ 8447.113594] Lustre: DEBUG MARKER: oleg158-server.virtnet: executing unload_modules_local [ 8449.631378] Key type lgssc unregistered [ 8449.850761] LNet: 276872:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 8449.857337] LNetError: 276872:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 8449.864827] LNet: Removed LNI 192.168.201.158@tcp [ 8450.410698] Key type .llcrypt unregistered [ 8450.412371] Key type ._llcrypt unregistered [ 8493.744610] Key type ._llcrypt registered [ 8493.747658] Key type .llcrypt registered [ 8493.856740] Lustre: DEBUG MARKER: oleg158-server.virtnet: executing load_modules_local [ 8494.682780] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 8494.693304] alg: No test for adler32 (adler32-zlib) [ 8495.822633] Lustre: Lustre: Build Version: 2.17.57_48_g018c4cb [ 8496.094392] LNet: Added LNI 192.168.201.158@tcp [8/256/0/180] [ 8497.767494] Key type lgssc registered [ 8499.123489] Lustre: Echo OBD driver; http://www.lustre.org/ [ 8512.618419] loop0: detected capacity change from 0 to 204800000 [ 8512.830855] blk_update_request: operation not supported error, dev loop0, sector 399872 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [ 8512.849095] blk_update_request: operation not supported error, dev loop0, sector 208 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [ 8513.064321] blk_update_request: operation not supported error, dev loop0, sector 160232 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [ 8513.901659] loop0: detected capacity change from 0 to 204800000 [ 8514.073229] blk_update_request: operation not supported error, dev loop0, sector 399872 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [ 8514.081827] blk_update_request: operation not supported error, dev loop0, sector 208 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [ 8514.490332] blk_update_request: operation not supported error, dev loop0, sector 160232 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [ 8515.456132] loop0: detected capacity change from 0 to 204800000 [ 8515.704677] blk_update_request: operation not supported error, dev loop0, sector 399872 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [ 8515.715127] blk_update_request: operation not supported error, dev loop0, sector 8224 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [ 8515.849636] blk_update_request: operation not supported error, dev loop0, sector 58264 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [ 8516.923934] loop0: detected capacity change from 0 to 204800000 [ 8517.106610] blk_update_request: operation not supported error, dev loop0, sector 399872 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [ 8518.307446] loop0: detected capacity change from 0 to 204800000 [ 8518.344523] LDISKFS-fs (loop0): mounted filesystem with ordered data mode. Opts: (null) [ 8522.704946] loop0: detected capacity change from 0 to 204800000 [ 8522.727762] LDISKFS-fs (loop0): mounted filesystem with ordered data mode. Opts: (null) [ 8526.929596] loop0: detected capacity change from 0 to 204800000 [ 8526.952938] LDISKFS-fs (loop0): mounted filesystem with ordered data mode. Opts: (null) [ 8531.021161] loop0: detected capacity change from 0 to 204800000 [ 8531.077875] LDISKFS-fs (loop0): mounted filesystem with ordered data mode. Opts: (null) [ 8535.156201] loop0: detected capacity change from 0 to 204800000 [ 8535.323249] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 8535.416162] LDISKFS-fs (loop0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 8535.564214] Lustre: 280286:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0000: restore index 'fld' with [0x200000001:0x3:0x0]: rc = 0 [ 8535.571256] Lustre: 280286:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0000: restore index '0x1010000-MDT0000' with [0x200000005:0x2:0x0]: rc = 0 [ 8535.588063] Lustre: 280286:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0000: restore index '0x1010000' with [0x200000003:0xb:0x0]: rc = 0 [ 8535.599613] Lustre: 280286:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0000: restore index '0x10000-MDT0000' with [0x200000005:0x1:0x0]: rc = 0 [ 8535.612250] Lustre: 280286:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0000: restore index '0x10000' with [0x200000003:0xa:0x0]: rc = 0 [ 8535.619191] Lustre: 280286:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0000: restore index '0x2010000' with [0x200000003:0xc:0x0]: rc = 0 [ 8535.639531] Lustre: 280286:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0000: restore index '0x2010000-MDT0000' with [0x200000005:0x3:0x0]: rc = 0 [ 8535.666961] Lustre: 280286:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0000: restore index 'nodemap' with [0x200000003:0x3:0x0]: rc = 0 [ 8535.679806] Lustre: 280286:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0000: restore index '0x1010000' with [0x200000006:0x1010000:0x0]: rc = 0 [ 8535.688566] Lustre: 280286:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0000: restore index '0x10000' with [0x200000006:0x10000:0x0]: rc = 0 [ 8535.696392] Lustre: 280286:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0000: restore index '0x2010000' with [0x200000006:0x2010000:0x0]: rc = 0 [ 8535.702533] Lustre: 280286:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0000: restore index '0x2020000' with [0x200000006:0x2020000:0x0]: rc = 0 [ 8535.711112] Lustre: 280286:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0000: restore index '0x20000' with [0x200000006:0x20000:0x0]: rc = 0 [ 8535.724494] Lustre: 280286:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0000: restore index '0x1020000' with [0x200000006:0x1020000:0x0]: rc = 0 [ 8541.824519] Lustre: server umount MGS complete [ 8542.901892] loop0: detected capacity change from 0 to 204800000 [ 8543.037765] LDISKFS-fs (loop0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 8543.540714] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 8543.613731] Lustre: lustre-MDT0000: Aborting client recovery [ 8544.541820] loop1: detected capacity change from 0 to 204800000 [ 8544.680536] LDISKFS-fs (loop1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 8544.800566] Lustre: 280719:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0001: restore index 'fld' with [0x200000001:0x3:0x0]: rc = 0 [ 8544.826757] Lustre: 280719:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0001: restore index '0x1010000' with [0x200000003:0x5:0x0]: rc = 0 [ 8544.855569] Lustre: 280719:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0001: restore index '0x2010000' with [0x200000003:0x6:0x0]: rc = 0 [ 8544.881880] Lustre: 280719:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0001: restore index '0x1010000-MDT0001' with [0x200000005:0x2:0x0]: rc = 0 [ 8544.888837] Lustre: 280719:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0001: restore index '0x10000' with [0x200000003:0x4:0x0]: rc = 0 [ 8544.896600] Lustre: 280719:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0001: restore index '0x2010000-MDT0001' with [0x200000005:0x3:0x0]: rc = 0 [ 8544.904069] Lustre: 280719:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0001: restore index '0x10000-MDT0001' with [0x200000005:0x1:0x0]: rc = 0 [ 8544.921429] Lustre: Found index 1 for lustre-MDT0001, updating log [ 8545.069938] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 8545.116849] Lustre: lustre-MDT0001: Aborting client recovery [ 8545.982627] loop2: detected capacity change from 0 to 204800000 [ 8546.224767] LDISKFS-fs (loop2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 8546.346792] Lustre: 280800:0:(osd_scrub.c:1854:osd_index_restore()) lustre-OST0000: restore index 'fld' with [0x200000001:0x3:0x0]: rc = 0 [ 8546.356800] Lustre: 280800:0:(osd_scrub.c:1854:osd_index_restore()) lustre-OST0000: restore index '0x20000' with [0x200000003:0x5:0x0]: rc = 0 [ 8546.368849] Lustre: 280800:0:(osd_scrub.c:1854:osd_index_restore()) lustre-OST0000: restore index '0x20000-OST0000' with [0x200000005:0x1:0x0]: rc = 0 [ 8546.376903] Lustre: 280800:0:(osd_scrub.c:1854:osd_index_restore()) lustre-OST0000: restore index '0x1020000' with [0x200000003:0x6:0x0]: rc = 0 [ 8546.393643] Lustre: 280800:0:(osd_scrub.c:1854:osd_index_restore()) lustre-OST0000: restore index '0x1020000-OST0000' with [0x200000005:0x2:0x0]: rc = 0 [ 8546.404735] Lustre: 280800:0:(osd_scrub.c:1854:osd_index_restore()) lustre-OST0000: restore index '0x2020000-OST0000' with [0x200000005:0x3:0x0]: rc = 0 [ 8546.417642] Lustre: 280800:0:(osd_scrub.c:1854:osd_index_restore()) lustre-OST0000: restore index '0x2020000' with [0x200000003:0x7:0x0]: rc = 0 [ 8546.579493] Lustre: Found index 0 for lustre-OST0000, updating log [ 8546.680277] Lustre: 280820:0:(osd_compat.c:1353:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 8546.789524] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 8546.835508] Lustre: lustre-OST0000: Aborting recovery [ 8547.693825] loop3: detected capacity change from 0 to 204800000 [ 8547.876933] LDISKFS-fs (loop3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 8547.970951] Lustre: 280885:0:(osd_scrub.c:1854:osd_index_restore()) lustre-OST0001: restore index 'fld' with [0x200000001:0x3:0x0]: rc = 0 [ 8547.981645] Lustre: 280885:0:(osd_scrub.c:1854:osd_index_restore()) lustre-OST0001: restore index '0x2020000-OST0001' with [0x200000005:0x3:0x0]: rc = 0 [ 8548.007785] Lustre: 280885:0:(osd_scrub.c:1854:osd_index_restore()) lustre-OST0001: restore index '0x1020000' with [0x200000003:0x6:0x0]: rc = 0 [ 8548.017615] Lustre: 280885:0:(osd_scrub.c:1854:osd_index_restore()) lustre-OST0001: restore index '0x1020000-OST0001' with [0x200000005:0x2:0x0]: rc = 0 [ 8548.043615] Lustre: 280885:0:(osd_scrub.c:1854:osd_index_restore()) lustre-OST0001: restore index '0x20000-OST0001' with [0x200000005:0x1:0x0]: rc = 0 [ 8548.061569] Lustre: 280885:0:(osd_scrub.c:1854:osd_index_restore()) lustre-OST0001: restore index '0x2020000' with [0x200000003:0x7:0x0]: rc = 0 [ 8548.068456] Lustre: 280885:0:(osd_scrub.c:1854:osd_index_restore()) lustre-OST0001: restore index '0x20000' with [0x200000003:0x5:0x0]: rc = 0 [ 8548.082133] Lustre: Found index 1 for lustre-OST0001, updating log [ 8548.095414] Lustre: 280892:0:(osd_compat.c:1353:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 8548.148300] Lustre: lustre-OST0001: Aborting recovery [ 8553.259731] Lustre: lustre-OST0001: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x0:6 to 0x0:33) [ 8553.263173] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x0:6 to 0x0:33) [ 8553.290940] Lustre: lustre-OST0001-osc-MDT0000: update sequence from 0x100010000 to 0x2c0000bd0 [ 8553.304586] LustreError: 280663:0:(lod_dev.c:511:lod_sub_recovery_thread()) lustre-MDT0001-osp-MDT0000: get update log duration 10, retries 0, failed: rc = -108 [ 8553.329723] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000bd0 [ 8555.305876] Lustre: lustre-OST0000: new connection from lustre-MDT0001-mdtlov (cleaning up unused objects from 0x280000400:5 to 0x280000400:33) [ 8555.311127] Lustre: lustre-OST0001: new connection from lustre-MDT0001-mdtlov (cleaning up unused objects from 0x2c0000400:5 to 0x2c0000400:33) [ 8556.121819] LustreError: 280630:0:(mdt_identity.c:115:mdt_identity_do_upcall()) lustre-MDT0000: error invoking upcall /root/Work/Lustre/R5/lustre-release/lustre/tests/../utils/l_getidentity lustre-MDT0000 0: rc -2; check /proc/fs/lustre/mdt/lustre-MDT0000/identity_upcall, time 1093us: rc = -2 [ 8556.632442] LustreError: 280627:0:(mdt_identity.c:115:mdt_identity_do_upcall()) lustre-MDT0000: error invoking upcall /root/Work/Lustre/R5/lustre-release/lustre/tests/../utils/l_getidentity lustre-MDT0000 0: rc -2; check /proc/fs/lustre/mdt/lustre-MDT0000/identity_upcall, time 340us: rc = -2 [ 8556.650710] LustreError: 280627:0:(mdt_identity.c:115:mdt_identity_do_upcall()) Skipped 19 previous similar messages [ 8570.847841] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 8570.854650] Lustre: lustre-MDT0000-lwp-OST0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 8570.860179] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 8570.898586] Lustre: Skipped 3 previous similar messages [ 8575.973251] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 8575.980037] Lustre: Skipped 6 previous similar messages [ 8576.129155] Lustre: server umount lustre-MDT0000 complete [ 8577.003909] LustreError: 280612:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1787801584 with bad export cookie 14772089539515970041 [ 8577.009389] LustreError: MGC192.168.201.158@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 8577.020784] LustreError: 280612:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 2 previous similar messages [ 8581.087988] LustreError: 280627:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 8581.088227] Lustre: lustre-MDT0001-lwp-OST0001: Connection to lustre-MDT0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 8581.091152] Lustre: lustre-MDT0001: Not available for connect from 0@lo (stopping) [ 8581.096993] LustreError: 280627:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 3 previous similar messages [ 8581.117565] Lustre: Skipped 1 previous similar message [ 8582.112240] LustreError: 280627:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 8582.125101] LustreError: 280627:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 1 previous similar message [ 8583.593392] Lustre: server umount lustre-MDT0001 complete [ 8590.820877] Lustre: server umount lustre-OST0000 complete [ 8606.176858] Lustre: lustre-OST0001 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 8606.449963] Lustre: server umount lustre-OST0001 complete [ 8614.568588] Lustre: DEBUG MARKER: == conf-sanity test 109a: test lctl clear_conf fsname ==== 23:33:39 (1787801619) [ 8640.073822] Lustre: DEBUG MARKER: oleg158-server.virtnet: executing set_hostid [ 8651.094657] Lustre: DEBUG MARKER: oleg158-server.virtnet: executing load_modules_local [ 8682.513595] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 8682.722147] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 8682.743477] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 8682.823492] Lustre: lustre-MDT0000: new disk, initializing [ 8682.881156] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 8682.883718] Lustre: Skipped 1 previous similar message [ 8682.894273] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 8687.329852] Lustre: DEBUG MARKER: oleg158-server.virtnet: executing set_default_debug -1 all [ 8696.747280] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 8696.871745] Lustre: 284956:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 8696.914437] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [ 8696.918089] Lustre: Skipped 1 previous similar message [ 8696.968753] Lustre: lustre-MDT0001: new disk, initializing [ 8697.045938] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 8697.085408] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [ 8697.096528] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 8700.772502] Lustre: DEBUG MARKER: oleg158-server.virtnet: executing set_default_debug -1 all [ 8709.938989] Lustre: DEBUG MARKER: oleg158-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 8715.284645] Lustre: DEBUG MARKER: oleg158-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 8722.047991] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 8722.360598] Lustre: lustre-OST0000: new disk, initializing [ 8722.366744] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 8722.376056] Lustre: 286926:0:(osd_compat.c:1353:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 8722.465271] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 8728.199710] Lustre: DEBUG MARKER: oleg158-server.virtnet: executing set_default_debug -1 all [ 8731.226107] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [ 8731.232905] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 8731.391235] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 8737.533953] Lustre: DEBUG MARKER: oleg158-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 8742.252928] Lustre: Setting parameter lustre-MDT0000.mdd.atime_diff=62 in log lustre-MDT0000 [ 8753.336275] Lustre: Modifying parameter lustre-MDT0000.mdd.atime_diff=63 in log lustre-MDT0000 [ 8757.389072] Lustre: Setting parameter lustre-client.llite.max_read_ahead_mb=32 in log lustre-client [ 8765.581652] Lustre: Modifying parameter lustre-client.llite.max_read_ahead_mb=64 in log lustre-client [ 8803.299957] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 8803.306327] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 8803.327101] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 8803.329835] Lustre: Skipped 3 previous similar messages [ 8805.728810] Lustre: server umount lustre-OST0000 complete [ 8809.955309] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 8809.969454] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 8809.989227] Lustre: Skipped 1 previous similar message [ 8809.995712] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 8810.003644] Lustre: Skipped 1 previous similar message [ 8818.656799] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 8818.661890] Lustre: Skipped 3 previous similar messages [ 8824.287585] Lustre: lustre-MDT0000 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 8824.414267] Lustre: server umount lustre-MDT0000 complete [ 8828.151339] LustreError: 284949:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1787801835 with bad export cookie 14772089539515972750 [ 8828.154618] LustreError: MGC192.168.201.158@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 8828.162124] LustreError: 284949:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 2 previous similar messages [ 8828.446577] Lustre: server umount lustre-MDT0001 complete [ 8837.040804] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 8842.553578] Lustre: DEBUG MARKER: oleg158-server.virtnet: executing set_default_debug -1 all [ 8848.862494] Lustre: server umount MGS complete [ 8860.932118] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 8861.449127] LustreError: 290742:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 8861.472200] LustreError: 290755:0:(qmt_pool.c:1639:qmt_pool_add_rem()) lustre-QMT0000: can't remove lustre-OST0000_UUID pool 'pool1': rc = -22 [ 8861.525581] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 8865.739415] Lustre: DEBUG MARKER: oleg158-server.virtnet: executing set_default_debug -1 all [ 8866.785851] LustreError: 290743:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 8871.905744] LustreError: 290742:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 8874.629299] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 8880.064199] Lustre: DEBUG MARKER: oleg158-server.virtnet: executing set_default_debug -1 all [ 8887.878919] Lustre: DEBUG MARKER: oleg158-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 8893.991290] Lustre: DEBUG MARKER: oleg158-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 8901.040282] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 8901.507142] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 8901.515722] Lustre: Skipped 1 previous similar message [ 8907.134734] Lustre: DEBUG MARKER: oleg158-server.virtnet: executing set_default_debug -1 all [ 8914.666855] Lustre: DEBUG MARKER: oleg158-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 8942.562612] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 8942.570745] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 8942.588939] Lustre: Skipped 1 previous similar message [ 8942.597599] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 8942.605416] Lustre: Skipped 2 previous similar messages [ 8948.701684] Lustre: server umount lustre-OST0000 complete [ 8955.889492] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 8955.910287] Lustre: Skipped 2 previous similar messages [ 8958.456737] Lustre: server umount lustre-MDT0000 complete [ 8960.993183] LustreError: 290739:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 8961.003578] LustreError: 290739:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 1 previous similar message [ 8962.167779] LustreError: 290723:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1787801969 with bad export cookie 14772089539515974577 [ 8962.182878] LustreError: MGC192.168.201.158@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 8962.674557] Lustre: server umount lustre-MDT0001 complete [ 8974.174462] Lustre: DEBUG MARKER: oleg158-server.virtnet: executing unload_modules_local [ 8977.431815] Key type lgssc unregistered [ 8977.951935] LNet: 294690:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 8977.980878] LNetError: 294690:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 8978.004446] LNet: Removed LNI 192.168.201.158@tcp [ 8979.008815] Key type .llcrypt unregistered [ 8979.013669] Key type ._llcrypt unregistered [ 9028.844144] Key type ._llcrypt registered [ 9028.848675] Key type .llcrypt registered [ 9028.949496] Lustre: DEBUG MARKER: oleg158-server.virtnet: executing set_hostid [ 9044.281427] Lustre: DEBUG MARKER: oleg158-server.virtnet: executing load_modules_local [ 9045.125305] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 9045.221038] alg: No test for adler32 (adler32-zlib) [ 9046.311324] Lustre: Lustre: Build Version: 2.17.57_48_g018c4cb [ 9046.492190] LNet: Added LNI 192.168.201.158@tcp [8/256/0/180] [ 9048.183173] Key type lgssc registered [ 9049.099339] Lustre: Echo OBD driver; http://www.lustre.org/ [ 9084.721146] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 9084.771477] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 9086.318908] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 9086.387079] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 9086.566892] Lustre: lustre-MDT0000: new disk, initializing [ 9086.694504] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 9086.710984] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 9091.309450] Lustre: DEBUG MARKER: oleg158-server.virtnet: executing set_default_debug -1 all [ 9105.105811] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 9105.174953] Lustre: 298662:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 9105.230612] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [ 9105.232409] Lustre: Skipped 1 previous similar message [ 9105.270294] Lustre: lustre-MDT0001: new disk, initializing [ 9105.331451] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 9105.355214] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [ 9105.366571] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 9111.303173] Lustre: DEBUG MARKER: oleg158-server.virtnet: executing set_default_debug -1 all [ 9121.115326] Lustre: DEBUG MARKER: oleg158-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 9126.886771] Lustre: DEBUG MARKER: oleg158-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 9132.993556] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 9133.311888] Lustre: lustre-OST0000: new disk, initializing [ 9133.316147] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 9133.322338] Lustre: 300631:0:(osd_compat.c:1353:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 9133.392963] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 9139.283687] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [ 9139.294780] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 9139.433774] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 9139.923238] Lustre: DEBUG MARKER: oleg158-server.virtnet: executing set_default_debug -1 all [ 9149.625585] Lustre: DEBUG MARKER: oleg158-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 9157.564572] Lustre: Setting parameter lustre-MDT0000.mdd.atime_diff=62 in log lustre-MDT0000 [ 9172.329463] Lustre: Modifying parameter lustre-MDT0000.mdd.atime_diff=63 in log lustre-MDT0000 [ 9176.369221] Lustre: Setting parameter lustre-client.llite.max_read_ahead_mb=32 in log lustre-client [ 9181.227447] Lustre: Modifying parameter lustre-client.llite.max_read_ahead_mb=64 in log lustre-client [ 9221.602292] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 9221.607074] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 9221.636233] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 9221.640355] LustreError: Skipped 1 previous similar message [ 9225.848648] Lustre: server umount lustre-OST0000 complete [ 9231.844330] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 9231.859449] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 9231.865471] Lustre: Skipped 2 previous similar messages [ 9231.879159] Lustre: Skipped 1 previous similar message [ 9235.971567] Lustre: server umount lustre-MDT0000 complete [ 9239.361257] LustreError: 298655:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1787802246 with bad export cookie 4835788314068159070 [ 9239.365453] LustreError: MGC192.168.201.158@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 9239.371703] LustreError: 298655:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 9239.679899] Lustre: server umount lustre-MDT0001 complete [ 9249.537050] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 9253.895178] Lustre: DEBUG MARKER: oleg158-server.virtnet: executing set_default_debug -1 all [ 9258.409288] Lustre: server umount MGS complete [ 9270.894261] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 9271.352476] LustreError: 304547:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 9271.376835] LustreError: 304547:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 3 previous similar messages [ 9271.378516] LustreError: 304560:0:(qmt_pool.c:1639:qmt_pool_add_rem()) lustre-QMT0000: can't remove lustre-OST0000_UUID pool 'pool1': rc = -22 [ 9271.462610] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 9275.450921] Lustre: DEBUG MARKER: oleg158-server.virtnet: executing set_default_debug -1 all [ 9276.896822] LustreError: 304548:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 9281.011302] LustreError: 304547:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 9284.702672] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 9285.090336] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 9289.476835] Lustre: DEBUG MARKER: oleg158-server.virtnet: executing set_default_debug -1 all [ 9297.616609] Lustre: DEBUG MARKER: oleg158-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 9303.271763] Lustre: DEBUG MARKER: oleg158-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 9310.229573] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 9310.654733] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 9319.014855] Lustre: DEBUG MARKER: oleg158-server.virtnet: executing set_default_debug -1 all [ 9327.865390] Lustre: DEBUG MARKER: oleg158-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 9357.796759] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 9357.805758] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 9357.828424] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 9359.846602] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 9359.852426] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 9359.888968] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 9362.800469] Lustre: server umount lustre-OST0000 complete [ 9367.008520] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 9367.023987] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 9367.052698] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 9367.059943] Lustre: Skipped 1 previous similar message [ 9373.111943] Lustre: server umount lustre-MDT0000 complete [ 9375.208189] LustreError: 304544:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 9375.240079] LustreError: 304544:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 1 previous similar message [ 9377.742636] LustreError: 304527:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1787802384 with bad export cookie 4835788314068160911 [ 9377.745463] LustreError: MGC192.168.201.158@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 9377.751075] LustreError: 304527:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 2 previous similar messages [ 9378.154317] Lustre: server umount lustre-MDT0001 complete [ 9393.427569] Lustre: DEBUG MARKER: oleg158-server.virtnet: executing unload_modules_local [ 9396.077612] Key type lgssc unregistered [ 9396.384950] LNet: 308495:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 9396.388300] LNetError: 308495:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 9397.412960] LNet: Removed LNI 192.168.201.158@tcp [ 9398.300638] Key type .llcrypt unregistered [ 9398.305743] Key type ._llcrypt unregistered [ 9422.335613] Key type ._llcrypt registered [ 9422.339075] Key type .llcrypt registered [ 9422.450288] Lustre: DEBUG MARKER: oleg158-server.virtnet: executing set_hostid [ 9441.672883] Lustre: DEBUG MARKER: oleg158-server.virtnet: executing load_modules_local [ 9443.048206] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 9443.195163] alg: No test for adler32 (adler32-zlib) [ 9444.229379] Lustre: Lustre: Build Version: 2.17.57_48_g018c4cb [ 9444.499395] LNet: Added LNI 192.168.201.158@tcp [8/256/0/180] [ 9446.169483] Key type lgssc registered [ 9447.281756] Lustre: Echo OBD driver; http://www.lustre.org/ [ 9476.880556] Lustre: DEBUG MARKER: oleg158-server.virtnet: executing load_modules_local [ 9491.851137] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 9491.923722] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 9493.244673] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 9493.293757] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 9493.404662] Lustre: lustre-MDT0000: new disk, initializing [ 9493.512939] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 9493.529931] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 9497.813043] Lustre: DEBUG MARKER: oleg158-server.virtnet: executing set_default_debug -1 all [ 9509.016221] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 9509.402232] Lustre: lustre-OST0000: new disk, initializing [ 9509.409210] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 9509.414982] Lustre: Skipped 1 previous similar message [ 9509.424616] Lustre: 312545:0:(osd_compat.c:1353:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 9509.515645] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 9510.514408] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:0:ost [ 9510.534097] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:0:ost] [ 9510.571322] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x240000400 [ 9516.314124] Lustre: DEBUG MARKER: oleg158-server.virtnet: executing set_default_debug -1 all [10355.169352] Lustre: lustre-MDT0000-lwp-OST0000: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [10355.192512] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [10358.559641] Lustre: server umount lustre-MDT0000 complete [10361.734909] LustreError: 311732:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1787803368 with bad export cookie 6702141793159448263 [10361.742836] LustreError: MGC192.168.201.158@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [10361.842424] Lustre: server umount lustre-OST0000 complete [10390.933717] Lustre: DEBUG MARKER: oleg158-server.virtnet: executing set_hostid [10404.894709] Lustre: DEBUG MARKER: oleg158-server.virtnet: executing load_modules_local [10455.718369] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [10456.156987] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [10456.201801] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [10456.397714] Lustre: lustre-MDT0000: new disk, initializing [10456.665671] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [10456.702524] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [10462.259511] Lustre: DEBUG MARKER: oleg158-server.virtnet: executing set_default_debug -1 all [10474.418455] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [10474.563308] Lustre: 316902:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [10474.629389] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [10474.634342] Lustre: Skipped 1 previous similar message [10474.753251] Lustre: lustre-MDT0001: new disk, initializing [10474.899261] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [10474.961077] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [10474.976323] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [10480.265984] Lustre: DEBUG MARKER: oleg158-server.virtnet: executing set_default_debug -1 all [10490.126942] Lustre: DEBUG MARKER: oleg158-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [10496.509422] Lustre: DEBUG MARKER: oleg158-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [10505.172378] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [10505.757889] Lustre: lustre-OST0000: new disk, initializing [10505.776605] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [10505.789706] Lustre: 318871:0:(osd_compat.c:1353:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [10506.023546] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [10512.489019] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [10512.549987] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [10512.749208] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [10513.737544] Lustre: DEBUG MARKER: oleg158-server.virtnet: executing set_default_debug -1 all [10530.135296] Lustre: DEBUG MARKER: oleg158-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [10538.116802] Lustre: DEBUG MARKER: oleg158-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 [10538.385982] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [10542.833021] Lustre: DEBUG MARKER: oleg158-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid 50 [10543.131333] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [10548.710432] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [10548.713392] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [10548.721782] LustreError: Skipped 1 previous similar message [10548.723801] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [10548.747571] Lustre: Skipped 1 previous similar message [10551.430404] Lustre: server umount lustre-OST0000 complete [10556.897715] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [10556.904851] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [10556.911903] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [10556.914403] Lustre: Skipped 1 previous similar message [10558.949142] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [10558.949162] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [10558.960291] Lustre: Skipped 1 previous similar message [10561.883204] Lustre: server umount lustre-MDT0000 complete [10565.773267] LustreError: 316894:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1787803572 with bad export cookie 6702141793162389082 [10565.790065] LustreError: 316894:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 2 previous similar messages [10565.808593] LustreError: MGC192.168.201.158@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [10566.321573] Lustre: server umount lustre-MDT0001 complete [10570.302917] Lustre: DEBUG MARKER: SKIP: conf-sanity test_111 skipping SLOW test 111 [10572.596392] Lustre: DEBUG MARKER: == conf-sanity test 112a: mount OST with no_create option ========================================================== 00:06:17 (1787803577) [10580.635554] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [10581.497772] LustreError: 321187:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [10581.533590] LustreError: 321187:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 3 previous similar messages [10581.606812] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [10586.613071] LustreError: 321188:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [10587.396807] Lustre: DEBUG MARKER: oleg158-server.virtnet: executing set_default_debug -1 all [10591.731553] LustreError: 321187:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [10596.841436] LustreError: 321188:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [10599.763868] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [10600.224432] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [10606.346268] Lustre: DEBUG MARKER: oleg158-server.virtnet: executing set_default_debug -1 all [10614.745991] Lustre: DEBUG MARKER: oleg158-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [10620.851681] Lustre: DEBUG MARKER: oleg158-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [10627.728834] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [10628.061816] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [10633.973096] Lustre: DEBUG MARKER: oleg158-server.virtnet: executing set_default_debug -1 all [10642.143696] Lustre: DEBUG MARKER: oleg158-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [10648.584168] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [10648.744726] Lustre: lustre-OST0001: new disk, initializing [10648.747930] Lustre: srv-lustre-OST0001: No data found on store. Initialize space. [10648.753246] Lustre: 324022:0:(osd_compat.c:1353:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [10648.836060] Lustre: lustre-OST0001: Imperative Recovery not enabled, recovery window 60-180 [10654.340696] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x00000002c0000400-0x0000000300000400]:1:ost [10654.352249] Lustre: cli-lustre-OST0001-super: Allocated super-sequence [0x00000002c0000400-0x0000000300000400]:1:ost] [10654.416635] LustreError: 324338:0:(osp_precreate.c:974:osp_precreate_cleanup_orphans()) lustre-OST0001-osc-MDT0001: cannot cleanup orphans: rc = -1 [10654.771525] Lustre: DEBUG MARKER: oleg158-server.virtnet: executing set_default_debug -1 all [10655.482846] LustreError: 324338:0:(osp_precreate.c:974:osp_precreate_cleanup_orphans()) lustre-OST0001-osc-MDT0001: cannot cleanup orphans: rc = -1 [10655.497111] LustreError: 324338:0:(osp_precreate.c:974:osp_precreate_cleanup_orphans()) Skipped 2 previous similar messages [10656.577769] LustreError: 324345:0:(osp_precreate.c:974:osp_precreate_cleanup_orphans()) lustre-OST0001-osc-MDT0000: cannot cleanup orphans: rc = -1 [10658.657556] LustreError: 324345:0:(osp_precreate.c:974:osp_precreate_cleanup_orphans()) lustre-OST0001-osc-MDT0000: cannot cleanup orphans: rc = -1 [10658.679416] LustreError: 324345:0:(osp_precreate.c:974:osp_precreate_cleanup_orphans()) Skipped 3 previous similar messages [10662.835881] LustreError: 324345:0:(osp_precreate.c:974:osp_precreate_cleanup_orphans()) lustre-OST0001-osc-MDT0000: cannot cleanup orphans: rc = -1 [10662.848975] LustreError: 324345:0:(osp_precreate.c:974:osp_precreate_cleanup_orphans()) Skipped 7 previous similar messages [10666.162326] Lustre: DEBUG MARKER: oleg158-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 [10666.321265] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [10671.077359] LustreError: 324338:0:(osp_precreate.c:974:osp_precreate_cleanup_orphans()) lustre-OST0001-osc-MDT0001: cannot cleanup orphans: rc = -1 [10671.084583] LustreError: 324338:0:(osp_precreate.c:974:osp_precreate_cleanup_orphans()) Skipped 14 previous similar messages [10672.363688] Lustre: DEBUG MARKER: oleg158-client.virtnet: executing wait_import_state (FULL|IDLE) osc.lustre-OST0000-osc-ffff9979d8763800.ost_server_uuid 50 [10673.925297] Lustre: DEBUG MARKER: osc.lustre-OST0000-osc-ffff9979d8763800.ost_server_uuid in FULL state after 0 sec [10679.969615] Lustre: DEBUG MARKER: oleg158-client.virtnet: executing wait_import_state (FULL|IDLE) osc.lustre-OST0001-osc-ffff9979d8763800.ost_server_uuid 50 [10681.524784] Lustre: DEBUG MARKER: osc.lustre-OST0001-osc-ffff9979d8763800.ost_server_uuid in FULL state after 0 sec [10683.463451] Lustre: lustre-OST0001-osc-MDT0000: update sequence from 0x100010000 to 0x2c0000401 [10698.720676] LustreError: lustre-OST0001-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [10698.734553] Lustre: lustre-OST0001-osc-MDT0000: Connection to lustre-OST0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [10698.747381] Lustre: lustre-OST0001: Not available for connect from 0@lo (stopping) [10703.434682] Lustre: server umount lustre-OST0001 complete [10705.936945] LustreError: 323009:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [10705.981091] LustreError: 323009:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 1 previous similar message [10713.059248] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [10713.070737] LustreError: Skipped 1 previous similar message [10713.074513] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [10713.083257] Lustre: Skipped 1 previous similar message [10713.085973] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [10713.092493] Lustre: Skipped 4 previous similar messages [10716.155613] LustreError: 323007:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [10716.168175] LustreError: 323007:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 4 previous similar messages [10718.216803] Lustre: server umount lustre-OST0000 complete [10730.278862] Lustre: server umount lustre-MDT0000 complete [10734.039526] LustreError: 321167:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1787803741 with bad export cookie 6702141793162389964 [10734.065202] LustreError: 321167:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [10734.077088] LustreError: MGC192.168.201.158@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [10734.592066] Lustre: server umount lustre-MDT0001 complete [10748.324800] Lustre: DEBUG MARKER: oleg158-server.virtnet: executing unload_modules_local [10753.590918] Key type lgssc unregistered [10753.891408] LNet: 326720:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [10753.908215] LNetError: 326720:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [10753.944827] LNet: Removed LNI 192.168.201.158@tcp [10755.232393] Key type .llcrypt unregistered [10755.235637] Key type ._llcrypt unregistered [10787.326379] Key type ._llcrypt registered [10787.333215] Key type .llcrypt registered [10787.508727] Lustre: DEBUG MARKER: oleg158-server.virtnet: executing load_modules_local [10788.583214] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [10788.780989] alg: No test for adler32 (adler32-zlib) [10789.782703] Lustre: Lustre: Build Version: 2.17.57_48_g018c4cb [10789.958901] LNet: Added LNI 192.168.201.158@tcp [8/256/0/180] [10791.615963] Key type lgssc registered [10792.671798] Lustre: Echo OBD driver; http://www.lustre.org/ [10801.673086] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [10801.689868] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [10803.209645] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [10809.062946] Lustre: DEBUG MARKER: oleg158-server.virtnet: executing set_default_debug -1 all [10813.436294] LustreError: 328147:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [10813.464266] LustreError: 328147:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 2 previous similar messages [10818.530419] LustreError: 328146:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [10819.104876] Lustre: DEBUG MARKER: oleg158-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [10823.654687] LustreError: 328147:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [10828.787602] LustreError: 328146:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [10829.417275] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [10830.319387] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [10837.674327] Lustre: DEBUG MARKER: oleg158-server.virtnet: executing set_default_debug -1 all [10850.626655] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [10851.056878] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [10855.171110] LustreError: 329828:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [10859.419237] Lustre: DEBUG MARKER: oleg158-server.virtnet: executing set_default_debug -1 all [10865.649267] LustreError: 329828:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [10865.676137] LustreError: 329828:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 1 previous similar message [10869.052258] Lustre: DEBUG MARKER: oleg158-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [10873.854784] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:36 to 0x280000401:65) [10876.317221] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [10876.539747] Lustre: lustre-OST0001: Imperative Recovery not enabled, recovery window 60-180 [10882.071484] Lustre: lustre-OST0001: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x2c0000401:3 to 0x2c0000401:33) [10883.713840] Lustre: DEBUG MARKER: oleg158-server.virtnet: executing set_default_debug -1 all [10892.246252] Lustre: DEBUG MARKER: oleg158-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0001-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [10899.316502] Lustre: DEBUG MARKER: oleg158-server.virtnet: executing wait_import_state (FULL|IDLE) os[cp].lustre-OST0001-osc-MDT0001.ost_server_uuid 50 [10899.663241] Lustre: DEBUG MARKER: os[cp].lustre-OST0001-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [11020.259312] Lustre: lustre-OST0001-osc-MDT0000: Connection to lustre-OST0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [11020.268666] Lustre: lustre-OST0001: Not available for connect from 0@lo (stopping) [11020.283054] LustreError: lustre-OST0001-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [11023.119620] Lustre: server umount lustre-OST0001 complete [11025.403717] LustreError: 330050:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [11025.428525] LustreError: 330050:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 4 previous similar messages [11032.553584] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [11032.565320] LustreError: Skipped 1 previous similar message [11032.571225] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [11032.585804] Lustre: Skipped 1 previous similar message [11032.595499] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [11032.597632] Lustre: Skipped 1 previous similar message [11034.592776] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [11034.604903] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [11034.629032] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [11036.966621] Lustre: server umount lustre-OST0000 complete [11040.738280] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [11040.761196] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [11040.770404] Lustre: Skipped 2 previous similar messages [11045.361620] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [11045.372979] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [11045.386364] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [11046.807120] Lustre: server umount lustre-MDT0000 complete [11050.428914] LustreError: 328126:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1787804057 with bad export cookie 10549036579384054887 [11050.436449] LustreError: 328126:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 1 previous similar message [11050.439272] LustreError: MGC192.168.201.158@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [11050.983559] Lustre: server umount lustre-MDT0001 complete [11067.746224] Lustre: DEBUG MARKER: oleg158-server.virtnet: executing unload_modules_local [11072.510967] Key type lgssc unregistered [11073.171958] LNet: 333099:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [11073.189089] LNetError: 333099:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [11074.214981] LNet: Removed LNI 192.168.201.158@tcp [11075.740176] Key type .llcrypt unregistered [11075.742771] Key type ._llcrypt unregistered [11122.096664] Key type ._llcrypt registered [11122.099702] Key type .llcrypt registered [11122.217403] Lustre: DEBUG MARKER: oleg158-server.virtnet: executing load_modules_local [11123.478533] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [11123.559948] alg: No test for adler32 (adler32-zlib) [11124.799763] Lustre: Lustre: Build Version: 2.17.57_48_g018c4cb [11125.036835] LNet: Added LNI 192.168.201.158@tcp [8/256/0/180] [11126.695171] Key type lgssc registered [11128.062043] Lustre: Echo OBD driver; http://www.lustre.org/ [11139.099634] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [11139.114979] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [11140.710907] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [11146.021990] Lustre: DEBUG MARKER: oleg158-server.virtnet: executing set_default_debug -1 all [11150.817723] LustreError: 335391:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [11150.844321] LustreError: 335391:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 2 previous similar messages [11155.622683] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [11155.942300] LustreError: 335392:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [11156.387400] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [11163.275688] Lustre: DEBUG MARKER: oleg158-server.virtnet: executing set_default_debug -1 all [11167.076957] Lustre: Modifying parameter general.debug_raw_pointers=Y in log params [11176.725775] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [11177.512331] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [11181.645456] LustreError: 336880:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [11185.383825] Lustre: DEBUG MARKER: oleg158-server.virtnet: executing set_default_debug -1 all [11185.768109] LustreError: 337266:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [11185.774177] LustreError: 337266:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 1 previous similar message [11185.787298] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:36 to 0x280000401:97) [11191.269313] LustreError: 336880:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [11191.292680] LustreError: 336880:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 1 previous similar message [11196.184384] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [11196.711691] Lustre: lustre-OST0001: Imperative Recovery not enabled, recovery window 60-180 [11202.078576] Lustre: lustre-OST0001: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x2c0000401:3 to 0x2c0000401:65) [11205.975945] Lustre: DEBUG MARKER: oleg158-server.virtnet: executing set_default_debug -1 all [11215.586657] Lustre: DEBUG MARKER: Using TIMEOUT=20 [11225.944956] Lustre: Setting parameter general.lod.*.mdt_hash=crush in log params [11242.988027] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [11243.010832] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [11243.014214] Lustre: Skipped 1 previous similar message [11243.043616] Lustre: Skipped 3 previous similar messages [11246.328234] Lustre: server umount lustre-MDT0000 complete [11248.100971] LustreError: 337884:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [11248.122154] LustreError: 337884:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 5 previous similar messages [11251.034409] LustreError: 335371:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1787804258 with bad export cookie 12121558354084579117 [11251.036699] LustreError: MGC192.168.201.158@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [11251.047984] LustreError: 335371:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [11251.560882] Lustre: server umount lustre-MDT0001 complete [11267.963221] Lustre: server umount lustre-OST0000 complete [11269.472361] Lustre: 334655:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1787804260/real 1787804260] req@ffffa03903625180 x1874648505585792/t0(0) o400->lustre-MDT0001-lwp-OST0001@0@lo:12/10 lens 224/224 e 0 to 1 dl 1787804276 ref 1 fl Rpc:XNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 [11269.492947] Lustre: lustre-MDT0001-lwp-OST0001: Connection to lustre-MDT0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [11269.500217] Lustre: Skipped 2 previous similar messages [11271.646333] Lustre: server umount lustre-OST0001 complete [11289.030628] Lustre: DEBUG MARKER: oleg158-server.virtnet: executing load_modules_local [11300.521098] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [11301.424167] LustreError: 340657:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [11301.606224] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [11309.628380] Lustre: DEBUG MARKER: oleg158-server.virtnet: executing set_default_debug -1 all [11322.817498] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [11323.294696] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [11329.181880] Lustre: DEBUG MARKER: oleg158-server.virtnet: executing set_default_debug -1 all [11332.143528] Lustre: 341798:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify general/debug_raw_pointers=Y (mode = 0) failed: rc = -17 [11339.197077] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [11339.586325] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [11345.592515] Lustre: DEBUG MARKER: oleg158-server.virtnet: executing set_default_debug -1 all [11346.742022] LustreError: 342152:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [11346.765590] LustreError: 342152:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 4 previous similar messages [11346.773404] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:36 to 0x280000401:129) [11354.430859] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [11359.747615] Lustre: lustre-OST0001: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x2c0000401:3 to 0x2c0000401:97) [11360.631749] Lustre: DEBUG MARKER: oleg158-server.virtnet: executing set_default_debug -1 all [11368.920976] Lustre: DEBUG MARKER: Using TIMEOUT=20 [11372.633976] Lustre: 343667:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify general/lod.*.mdt_hash=crush (mode = 0) failed: rc = -17 [11379.686575] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [11379.697979] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [11379.719124] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [11382.651127] Lustre: server umount lustre-MDT0000 complete [11385.872878] LustreError: 340639:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1787804392 with bad export cookie 12121558354084580818 [11385.888635] LustreError: MGC192.168.201.158@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [11385.893441] LustreError: 340639:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 2 previous similar messages [11386.109658] Lustre: server umount lustre-MDT0001 complete [11399.781890] Lustre: server umount lustre-OST0000 complete [11413.893265] Lustre: server umount lustre-OST0001 complete [11418.193952] Lustre: DEBUG MARKER: SKIP: conf-sanity test_114 skipping SLOW test 114 [11419.913944] Lustre: DEBUG MARKER: SKIP: conf-sanity test_115 skipping excluded test 115 [11422.482249] Lustre: DEBUG MARKER: == conf-sanity test 116: big size MDT support ============ 00:20:27 (1787804427) [11449.639810] Lustre: DEBUG MARKER: oleg158-server.virtnet: executing load_modules_local [11457.248547] loop0: detected capacity change from 0 to 1099511627776 [11457.687917] SGI XFS with ACLs, security attributes, quota, no debug enabled [11457.693401] *** VALIDATE xfs *** [11457.704751] XFS (loop0): Mounting V5 Filesystem [11457.719956] XFS (loop0): Ending clean mount [11458.852702] loop1: detected capacity change from 0 to 18691697672192 [11552.900927] Lustre: DEBUG MARKER: oleg158-server.virtnet: executing set_hostid [11561.320379] Lustre: DEBUG MARKER: oleg158-server.virtnet: executing load_modules_local [11591.409917] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [11591.636770] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [11591.659609] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [11591.734765] Lustre: lustre-MDT0000: new disk, initializing [11591.788314] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [11591.793833] Lustre: Skipped 1 previous similar message [11591.806772] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [11595.205656] Lustre: DEBUG MARKER: oleg158-server.virtnet: executing set_default_debug -1 all [11603.996430] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [11604.102505] Lustre: 348800:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [11604.141806] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [11604.145152] Lustre: Skipped 1 previous similar message [11604.195454] Lustre: lustre-MDT0001: new disk, initializing [11604.296310] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [11604.306433] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [11607.942942] Lustre: DEBUG MARKER: oleg158-server.virtnet: executing set_default_debug -1 all [11615.661956] Lustre: DEBUG MARKER: oleg158-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [11620.998767] Lustre: DEBUG MARKER: oleg158-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [11626.380791] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [11626.607240] Lustre: lustre-OST0000: new disk, initializing [11626.611503] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [11626.621077] Lustre: 350769:0:(osd_compat.c:1353:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [11628.354391] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [11628.362633] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [11628.473643] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [11631.487426] Lustre: DEBUG MARKER: oleg158-server.virtnet: executing set_default_debug -1 all [11640.265398] Lustre: DEBUG MARKER: oleg158-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [11644.974166] Lustre: DEBUG MARKER: oleg158-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 [11645.180483] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [11649.482048] Lustre: DEBUG MARKER: oleg158-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid 50 [11649.670070] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [11654.114272] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [11654.128265] Lustre: Skipped 3 previous similar messages [11654.139524] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [11654.141275] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [11654.156138] Lustre: Skipped 4 previous similar messages [11657.807751] Lustre: server umount lustre-OST0000 complete [11664.353853] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [11664.364684] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [11664.366670] Lustre: Skipped 2 previous similar messages [11664.384332] Lustre: Skipped 2 previous similar messages [11667.423600] Lustre: server umount lustre-MDT0000 complete [11669.475946] LustreError: 351232:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [11669.494687] LustreError: 351232:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 9 previous similar messages [11670.170831] LustreError: 350768:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1787804677 with bad export cookie 12121558354084582372 [11670.174370] LustreError: MGC192.168.201.158@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [11670.182414] LustreError: 350768:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 2 previous similar messages [11670.468158] Lustre: server umount lustre-MDT0001 complete [11677.397041] XFS (loop0): Unmounting Filesystem [11681.668541] Lustre: DEBUG MARKER: == conf-sanity test 117: lctl get_param return errors properly ========================================================== 00:24:47 (1787804687) [11687.327547] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [11687.761290] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [11687.766581] Lustre: Skipped 2 previous similar messages [11691.465903] Lustre: DEBUG MARKER: oleg158-server.virtnet: executing set_default_debug -1 all [11697.853865] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [11701.083505] Lustre: DEBUG MARKER: oleg158-server.virtnet: executing set_default_debug -1 all [11708.629877] Lustre: DEBUG MARKER: oleg158-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [11713.561706] Lustre: DEBUG MARKER: oleg158-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [11720.435947] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [11727.492185] Lustre: DEBUG MARKER: oleg158-server.virtnet: executing set_default_debug -1 all [11733.605470] Lustre: DEBUG MARKER: oleg158-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [11742.693459] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [11742.695420] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [11742.696276] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [11742.720298] Lustre: Skipped 1 previous similar message [11746.992543] Lustre: server umount lustre-OST0000 complete [11752.933623] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [11752.940579] Lustre: Skipped 2 previous similar messages [11755.906631] Lustre: server umount lustre-MDT0000 complete [11759.085210] LustreError: 353308:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1787804766 with bad export cookie 12121558354084583254 [11759.090392] LustreError: MGC192.168.201.158@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [11759.095085] LustreError: 353308:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 3 previous similar messages [11759.363133] Lustre: server umount lustre-MDT0001 complete [11770.462783] Lustre: DEBUG MARKER: oleg158-server.virtnet: executing unload_modules_local [11773.390588] Key type lgssc unregistered [11773.682834] LNet: 356958:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [11773.689910] LNetError: 356958:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [11773.705424] LNet: Removed LNI 192.168.201.158@tcp [11774.532155] Key type .llcrypt unregistered [11774.537835] Key type ._llcrypt unregistered [11800.345380] Key type ._llcrypt registered [11800.347367] Key type .llcrypt registered [11800.439703] Lustre: DEBUG MARKER: oleg158-server.virtnet: executing load_modules_local [11801.659796] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [11801.705833] alg: No test for adler32 (adler32-zlib) [11802.697667] Lustre: Lustre: Build Version: 2.17.57_48_g018c4cb [11802.856377] LNet: Added LNI 192.168.201.158@tcp [8/256/0/180] [11804.487182] Key type lgssc registered [11805.787462] Lustre: Echo OBD driver; http://www.lustre.org/ [11812.393685] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [11812.404628] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [11813.814425] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [11818.420667] Lustre: DEBUG MARKER: oleg158-server.virtnet: executing set_default_debug -1 all [11823.078421] LustreError: 358524:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [11823.092313] LustreError: 358524:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 2 previous similar messages [11825.460600] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [11825.708095] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [11829.180681] Lustre: DEBUG MARKER: oleg158-server.virtnet: executing set_default_debug -1 all [11836.532082] Lustre: DEBUG MARKER: oleg158-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [11841.504366] Lustre: DEBUG MARKER: oleg158-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [11849.049422] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [11849.385348] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [11856.295682] Lustre: DEBUG MARKER: oleg158-server.virtnet: executing set_default_debug -1 all [11864.354502] Lustre: DEBUG MARKER: oleg158-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [11870.998774] Lustre: lustre-MDT0001: Not available for connect from 192.168.201.58@tcp (stopping) [11871.721138] LustreError: lustre-MDT0001-osp-MDT0000: operation mds_statfs to node 0@lo failed: rc = -107 [11871.727045] Lustre: lustre-MDT0001-osp-MDT0000: Connection to lustre-MDT0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [11871.742581] Lustre: lustre-MDT0001: Not available for connect from 0@lo (stopping) [11874.272723] Lustre: lustre-MDT0001: Not available for connect from 0@lo (stopping) [11874.273171] Lustre: lustre-MDT0001-lwp-OST0000: Connection to lustre-MDT0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [11876.587238] Lustre: server umount lustre-MDT0001 complete [11879.400161] LustreError: 358523:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [11879.418121] LustreError: 358523:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 1 previous similar message [11881.228728] LustreError: 358520:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 192.168.201.58@tcp (no target). If you are running an HA pair check that the target is mounted on the other server. [11884.519173] LustreError: 360435:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [11884.550170] LustreError: 360435:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 1 previous similar message [11885.837750] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [11885.919969] Lustre: MGS: Regenerating lustre-MDT0001 log by user request: rc = 0 [11885.930300] Lustre: Found index 1 for lustre-MDT0001, updating log [11885.967501] Lustre: 358511:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [11886.183228] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [11890.207654] Lustre: DEBUG MARKER: oleg158-server.virtnet: executing set_default_debug -1 all [11891.694201] Lustre: lustre-MDT0001-lwp-OST0000: Connection restored to 0@lo (at 0@lo) [11892.751178] LustreError: 362094:0:(obd_mount.c:234:lustre_start_simple()) lustre-MDT0001-lwp-OST0000 attach error -17 [11892.764987] LustreError: 362094:0:(tgt_mount.c:675:lustre_lwp_setup()) lustre-MDT0001-lwp-OST0000: setup up failed: rc -17 [11892.780283] LustreError: 358515:0:(mgc_request.c:614:do_requeue()) failed processing log: -17 [11892.850209] LustreError: 362101:0:(obd_config.c:2102:class_config_llog_handler()) MGC192.168.201.158@tcp: cfg command failed: rc = -17 [11892.860072] Lustre: cmd=cf001 0:lustre-MDT0001-osp-MDT0000 1:osp 2:lustre-MDT0000-mdtlov_UUID [11892.860072] [11901.738316] Lustre: lustre-MDT0001: Not available for connect from 192.168.201.58@tcp (stopping) [11901.749725] Lustre: Skipped 2 previous similar messages [11901.930403] Lustre: lustre-MDT0001-lwp-OST0000: Connection to lustre-MDT0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [11901.934426] LustreError: lustre-MDT0001-osp-MDT0000: operation mds_statfs to node 0@lo failed: rc = -107 [11906.155437] Lustre: server umount lustre-MDT0001 complete [11906.818993] LustreError: 358519:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 192.168.201.58@tcp (no target). If you are running an HA pair check that the target is mounted on the other server. [11916.925319] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [11917.028143] Lustre: MGS: Regenerating lustre-MDT0001 log by user request: rc = 0 [11917.031876] Lustre: Found index 1 for lustre-MDT0001, updating log [11917.055745] LustreError: 358519:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 192.168.201.58@tcp (no target). If you are running an HA pair check that the target is mounted on the other server. [11917.085675] LustreError: 358519:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 5 previous similar messages [11917.162129] Lustre: 358511:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [11917.731508] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [11922.938877] Lustre: lustre-MDT0001-lwp-OST0000: Connection restored to 0@lo (at 0@lo) [11924.445663] Lustre: DEBUG MARKER: oleg158-server.virtnet: executing set_default_debug -1 all [11925.510670] LustreError: 363108:0:(obd_mount.c:234:lustre_start_simple()) lustre-MDT0001-lwp-OST0000 attach error -17 [11925.522799] LustreError: 363108:0:(tgt_mount.c:675:lustre_lwp_setup()) lustre-MDT0001-lwp-OST0000: setup up failed: rc -17 [11925.531519] LustreError: 358515:0:(mgc_request.c:614:do_requeue()) failed processing log: -17 [11925.536925] LustreError: 358515:0:(mgc_request.c:614:do_requeue()) Skipped 1 previous similar message [11925.602214] LustreError: 363116:0:(obd_config.c:761:class_setup()) osp: device 8 already setup: rc = -17 [11925.607276] LustreError: 363116:0:(obd_config.c:866:class_detach()) OBD device 8 still set up [11925.611567] LustreError: 363116:0:(obd_config.c:2102:class_config_llog_handler()) MGC192.168.201.158@tcp: cfg command failed: rc = -17 [11925.617859] Lustre: cmd=cf003 0:lustre-MDT0001-osp-MDT0000 1:lustre-MDT0001_UUID 2:192.168.201.158@tcp [11925.617859] [12229.405930] Lustre: lustre-MDT0001: Not available for connect from 192.168.201.58@tcp (stopping) [12229.413608] Lustre: Skipped 2 previous similar messages [12230.113033] LustreError: lustre-MDT0001-osp-MDT0000: operation mds_statfs to node 0@lo failed: rc = -107 [12230.119630] Lustre: lustre-MDT0001-osp-MDT0000: Connection to lustre-MDT0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [12230.132973] Lustre: Skipped 2 previous similar messages [12235.757495] Lustre: server umount lustre-MDT0001 complete [12239.626854] LustreError: 358519:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 192.168.201.58@tcp (no target). If you are running an HA pair check that the target is mounted on the other server. [12239.643288] LustreError: 358519:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 2 previous similar messages [12244.621581] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [12244.685939] Lustre: MGS: Regenerating lustre-MDT0001 log by user request: rc = 0 [12244.689926] Lustre: Found index 1 for lustre-MDT0001, updating log [12244.718516] Lustre: 358512:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [12245.086626] Lustre: lustre-MDT0001: Imperative Recovery enabled, recovery window shrunk from 60-180 down to 60-180 [12249.216341] Lustre: DEBUG MARKER: oleg158-server.virtnet: executing set_default_debug -1 all [12250.094516] Lustre: lustre-MDT0001-lwp-OST0000: Connection restored to 0@lo (at 0@lo) [12251.127690] LustreError: 371580:0:(obd_mount.c:234:lustre_start_simple()) lustre-MDT0001-lwp-OST0000 attach error -17 [12251.135625] LustreError: 371580:0:(tgt_mount.c:675:lustre_lwp_setup()) lustre-MDT0001-lwp-OST0000: setup up failed: rc -17 [12251.147992] LustreError: 358515:0:(mgc_request.c:614:do_requeue()) failed processing log: -17 [12251.152708] LustreError: 358515:0:(mgc_request.c:614:do_requeue()) Skipped 1 previous similar message [12251.235610] LustreError: 371588:0:(lod_lov.c:208:lod_add_device()) lustre-MDT0001-osp-MDT0000: cannot connect to next dev lustre-MDT0001_UUID (-114) [12251.247495] LustreError: 371588:0:(obd_config.c:2102:class_config_llog_handler()) MGC192.168.201.158@tcp: cfg command failed: rc = -114 [12251.252683] Lustre: cmd=cf014 0:lustre-MDT0000-mdtlov 1:lustre-MDT0001_UUID 2:1 3:1 [12251.252683] [12571.617808] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [12571.636424] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [12571.644982] Lustre: Skipped 2 previous similar messages [12571.652742] Lustre: Skipped 7 previous similar messages [12575.601397] Lustre: server umount lustre-MDT0000 complete [12576.737621] LustreError: 358518:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [12576.763561] LustreError: 358518:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 5 previous similar messages [12578.874305] LustreError: 360365:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1787805585 with bad export cookie 15961836558562376019 [12578.878624] LustreError: MGC192.168.201.158@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [12578.884316] LustreError: 360365:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 1 previous similar message [12581.871303] Lustre: lustre-MDT0001-lwp-OST0000: Connection to lustre-MDT0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [12585.328478] Lustre: server umount lustre-MDT0001 complete [12588.760679] Lustre: server umount lustre-OST0000 complete [12597.843823] Lustre: DEBUG MARKER: oleg158-server.virtnet: executing set_hostid [12607.372395] Lustre: DEBUG MARKER: oleg158-server.virtnet: executing load_modules_local [12637.288968] Lustre: DEBUG MARKER: == conf-sanity test 120: cross-target rename should not create bad symlinks ========================================================== 00:40:43 (1787805643) [12643.006662] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [12643.194105] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [12643.211647] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [12643.251404] Lustre: lustre-MDT0000: new disk, initializing [12643.292120] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [12643.304329] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [12647.150930] Lustre: DEBUG MARKER: oleg158-server.virtnet: executing set_default_debug -1 all [12656.554833] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [12656.639151] Lustre: 383353:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [12656.684975] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [12656.688121] Lustre: Skipped 1 previous similar message [12656.741276] Lustre: lustre-MDT0001: new disk, initializing [12656.812946] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [12656.850337] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [12656.861301] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [12661.217754] Lustre: DEBUG MARKER: oleg158-server.virtnet: executing set_default_debug -1 all [12669.859579] Lustre: DEBUG MARKER: oleg158-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [12674.521393] Lustre: DEBUG MARKER: oleg158-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [12681.353713] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [12681.632868] Lustre: lustre-OST0000: new disk, initializing [12681.637786] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [12681.646992] Lustre: 385322:0:(osd_compat.c:1353:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [12681.713496] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [12687.580748] Lustre: DEBUG MARKER: oleg158-server.virtnet: executing set_default_debug -1 all [12690.063246] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [12690.075367] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [12690.163830] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [12698.165838] Lustre: DEBUG MARKER: oleg158-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [12705.764433] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [12705.768584] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [12705.776943] Lustre: Skipped 1 previous similar message [12705.789741] Lustre: Skipped 4 previous similar messages [12707.877351] Lustre: server umount lustre-MDT0000 complete [12710.882506] LustreError: 384864:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [12710.893542] LustreError: 384864:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 4 previous similar messages [12711.353681] LustreError: 383348:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1787805718 with bad export cookie 15961836558562377972 [12711.357508] LustreError: MGC192.168.201.158@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [12711.363367] LustreError: 383348:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 2 previous similar messages [12711.380958] LustreError: 384160:0:(osp_object.c:618:osp_attr_get()) lustre-MDT0000-osp-MDT0001: osp_attr_get update error [0x200000401:0x2:0x0]: rc = -5 [12711.743279] Lustre: server umount lustre-MDT0001 complete [12726.431513] Lustre: server umount lustre-OST0000 complete [12755.780587] Lustre: DEBUG MARKER: oleg158-server.virtnet: executing set_hostid [12767.912808] Lustre: DEBUG MARKER: oleg158-server.virtnet: executing load_modules_local [12800.432674] Lustre: DEBUG MARKER: == conf-sanity test complete, duration 12497 sec ========= 00:43:25 (1787805805) [12802.315387] Lustre: DEBUG MARKER: === conf-sanity: start cleanup 00:43:27 (1787805807) === [12806.350840] Lustre: DEBUG MARKER: === conf-sanity: finish cleanup 00:43:31 (1787805811) === [12850.482310] Lustre: DEBUG MARKER: oleg158-server.virtnet: executing unload_modules_local [12853.664254] Key type lgssc unregistered [12853.992660] LNet: 391694:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [12853.998296] LNetError: 391694:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [12854.016382] LNet: Removed LNI 192.168.201.158@tcp [12854.837156] Key type .llcrypt unregistered [12854.838495] Key type ._llcrypt unregistered