[ 0.000000] Linux version 4.18.0rh8.10-debug (green@maintenance) (gcc version 8.5.0 20210514 (Red Hat 8.5.0-26) (GCC)) #2 SMP Mon Jul 14 01:24:22 EDT 2025 [ 0.000000] Command line: rd.shell root=nbd:192.168.200.253:rocky8.10:ext4:ro:-p,-b4096 ro crashkernel=256M panic=1 nomodeset ipmtu=9000 ip=dhcp rd.neednet=1 init_on_free=off mitigations=off console=ttyS1,115200 audit=0 [ 0.000000] x86/fpu: Supporting XSAVE feature 0x001: 'x87 floating point registers' [ 0.000000] x86/fpu: Supporting XSAVE feature 0x002: 'SSE registers' [ 0.000000] x86/fpu: Supporting XSAVE feature 0x004: 'AVX registers' [ 0.000000] x86/fpu: xstate_offset[2]: 576, xstate_sizes[2]: 256 [ 0.000000] x86/fpu: Enabled xstate features 0x7, context size is 832 bytes, using 'standard' format. [ 0.000000] signal: max sigframe size: 1776 [ 0.000000] BIOS-provided physical RAM map: [ 0.000000] BIOS-e820: [mem 0x0000000000000000-0x000000000009fbff] usable [ 0.000000] BIOS-e820: [mem 0x000000000009fc00-0x000000000009ffff] reserved [ 0.000000] BIOS-e820: [mem 0x00000000000f0000-0x00000000000fffff] reserved [ 0.000000] BIOS-e820: [mem 0x0000000000100000-0x00000000bffcdfff] usable [ 0.000000] BIOS-e820: [mem 0x00000000bffce000-0x00000000bfffffff] reserved [ 0.000000] BIOS-e820: [mem 0x00000000feffc000-0x00000000feffffff] reserved [ 0.000000] BIOS-e820: [mem 0x00000000fffc0000-0x00000000ffffffff] reserved [ 0.000000] BIOS-e820: [mem 0x0000000100000000-0x0000000146dfffff] usable [ 0.000000] NX (Execute Disable) protection: active [ 0.000000] SMBIOS 2.8 present. [ 0.000000] DMI: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.17.0-10.fc44 06/10/2025 [ 0.000000] Hypervisor detected: KVM [ 0.000000] kvm-clock: Using msrs 4b564d01 and 4b564d00 [ 0.000000] kvm-clock: using sched offset of 525178097 cycles [ 0.000000] clocksource: kvm-clock: mask: 0xffffffffffffffff max_cycles: 0x1cd42e4dffb, max_idle_ns: 881590591483 ns [ 0.000000] tsc: Detected 2399.998 MHz processor [ 0.000000] last_pfn = 0x146e00 max_arch_pfn = 0x400000000 [ 0.000000] x86/PAT: Configuration [0-7]: WB WC UC- UC WB WP UC- WT [ 0.000000] last_pfn = 0xbffce max_arch_pfn = 0x400000000 [ 0.000000] found SMP MP-table at [mem 0x000f54b0-0x000f54bf] [ 0.000000] RAMDISK: [mem 0xbcc54000-0xbffbffff] [ 0.000000] ACPI: Early table checksum verification disabled [ 0.000000] ACPI: RSDP 0x00000000000F52D0 000014 (v00 BOCHS ) [ 0.000000] ACPI: RSDT 0x00000000BFFE247C 000034 (v01 BOCHS BXPC 00000001 BXPC 00000001) [ 0.000000] ACPI: FACP 0x00000000BFFE2318 000074 (v01 BOCHS BXPC 00000001 BXPC 00000001) [ 0.000000] ACPI: DSDT 0x00000000BFFE0040 0022D8 (v01 BOCHS BXPC 00000001 BXPC 00000001) [ 0.000000] ACPI: FACS 0x00000000BFFE0000 000040 [ 0.000000] ACPI: APIC 0x00000000BFFE238C 000090 (v03 BOCHS BXPC 00000001 BXPC 00000001) [ 0.000000] ACPI: HPET 0x00000000BFFE241C 000038 (v01 BOCHS BXPC 00000001 BXPC 00000001) [ 0.000000] ACPI: WAET 0x00000000BFFE2454 000028 (v01 BOCHS BXPC 00000001 BXPC 00000001) [ 0.000000] ACPI: Reserving FACP table memory at [mem 0xbffe2318-0xbffe238b] [ 0.000000] ACPI: Reserving DSDT table memory at [mem 0xbffe0040-0xbffe2317] [ 0.000000] ACPI: Reserving FACS table memory at [mem 0xbffe0000-0xbffe003f] [ 0.000000] ACPI: Reserving APIC table memory at [mem 0xbffe238c-0xbffe241b] [ 0.000000] ACPI: Reserving HPET table memory at [mem 0xbffe241c-0xbffe2453] [ 0.000000] ACPI: Reserving WAET table memory at [mem 0xbffe2454-0xbffe247b] [ 0.000000] No NUMA configuration found [ 0.000000] Faking a node at [mem 0x0000000000000000-0x0000000146dfffff] [ 0.000000] NODE_DATA(0) allocated [mem 0x1465a3000-0x1465cdfff] [ 0.000000] Reserving 256MB of memory at 2752MB for crashkernel (System RAM: 4205MB) [ 0.000000] Zone ranges: [ 0.000000] DMA [mem 0x0000000000001000-0x0000000000ffffff] [ 0.000000] DMA32 [mem 0x0000000001000000-0x00000000ffffffff] [ 0.000000] Normal [mem 0x0000000100000000-0x0000000146dfffff] [ 0.000000] Device empty [ 0.000000] Movable zone start for each node [ 0.000000] Early memory node ranges [ 0.000000] node 0: [mem 0x0000000000001000-0x000000000009efff] [ 0.000000] node 0: [mem 0x0000000000100000-0x00000000bffcdfff] [ 0.000000] node 0: [mem 0x0000000100000000-0x0000000146dfffff] [ 0.000000] Zeroed struct page in unavailable ranges: 4756 pages [ 0.000000] Initmem setup node 0 [mem 0x0000000000001000-0x0000000146dfffff] [ 0.000000] ACPI: PM-Timer IO Port: 0x608 [ 0.000000] ACPI: LAPIC_NMI (acpi_id[0xff] dfl dfl lint[0x1]) [ 0.000000] IOAPIC[0]: apic_id 0, version 17, address 0xfec00000, GSI 0-23 [ 0.000000] ACPI: INT_SRC_OVR (bus 0 bus_irq 0 global_irq 2 dfl dfl) [ 0.000000] ACPI: INT_SRC_OVR (bus 0 bus_irq 5 global_irq 5 high level) [ 0.000000] ACPI: INT_SRC_OVR (bus 0 bus_irq 9 global_irq 9 high level) [ 0.000000] ACPI: INT_SRC_OVR (bus 0 bus_irq 10 global_irq 10 high level) [ 0.000000] ACPI: INT_SRC_OVR (bus 0 bus_irq 11 global_irq 11 high level) [ 0.000000] Using ACPI (MADT) for SMP configuration information [ 0.000000] ACPI: HPET id: 0x8086a201 base: 0xfed00000 [ 0.000000] TSC deadline timer available [ 0.000000] smpboot: Allowing 4 CPUs, 0 hotplug CPUs [ 0.000000] kvm-guest: KVM setup pv remote TLB flush [ 0.000000] kvm-guest: setup PV sched yield [ 0.000000] PM: Registered nosave memory: [mem 0x00000000-0x00000fff] [ 0.000000] PM: Registered nosave memory: [mem 0x0009f000-0x0009ffff] [ 0.000000] PM: Registered nosave memory: [mem 0x000a0000-0x000effff] [ 0.000000] PM: Registered nosave memory: [mem 0x000f0000-0x000fffff] [ 0.000000] PM: Registered nosave memory: [mem 0xbffce000-0xbfffffff] [ 0.000000] PM: Registered nosave memory: [mem 0xc0000000-0xfeffbfff] [ 0.000000] PM: Registered nosave memory: [mem 0xfeffc000-0xfeffffff] [ 0.000000] PM: Registered nosave memory: [mem 0xff000000-0xfffbffff] [ 0.000000] PM: Registered nosave memory: [mem 0xfffc0000-0xffffffff] [ 0.000000] [mem 0xc0000000-0xfeffbfff] available for PCI devices [ 0.000000] Booting paravirtualized kernel on KVM [ 0.000000] clocksource: refined-jiffies: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 1910969940391419 ns [ 0.000000] setup_percpu: NR_CPUS:8192 nr_cpumask_bits:4 nr_cpu_ids:4 nr_node_ids:1 [ 0.000000] percpu: Embedded 63 pages/cpu s221184 r8192 d28672 u524288 [ 0.000000] kvm-guest: PV spinlocks enabled [ 0.000000] PV qspinlock hash table entries: 256 (order: 0, 4096 bytes, linear) [ 0.000000] Built 1 zonelists, mobility grouping on. Total pages: 1059606 [ 0.000000] Policy zone: Normal [ 0.000000] Kernel command line: rd.shell root=nbd:192.168.200.253:rocky8.10:ext4:ro:-p,-b4096 ro crashkernel=256M panic=1 nomodeset ipmtu=9000 ip=dhcp rd.neednet=1 init_on_free=off mitigations=off console=ttyS1,115200 audit=0 [ 0.000000] Specific versions of hardware are certified with Red Hat Enterprise Linux 8. Please see the list of hardware certified with Red Hat Enterprise Linux 8 at https://catalog.redhat.com. [ 0.000000] audit: disabled (until reboot) [ 0.000000] software IO TLB: area num 4. [ 0.000000] Memory: 2829652K/4306352K available (18435K kernel code, 11221K rwdata, 7248K rodata, 2908K init, 18040K bss, 524580K reserved, 0K cma-reserved) [ 0.000000] SLUB: HWalign=64, Order=0-3, MinObjects=0, CPUs=4, Nodes=1 [ 0.000000] kmemleak: Kernel memory leak detector disabled [ 0.000000] ftrace: allocating 41240 entries in 162 pages [ 0.000000] ftrace: allocated 162 pages with 3 groups [ 0.000000] rcu: Hierarchical RCU implementation. [ 0.000000] rcu: RCU event tracing is enabled. [ 0.000000] rcu: RCU restricting CPUs from NR_CPUS=8192 to nr_cpu_ids=4. [ 0.000000] rcu: RCU callback double-/use-after-free debug enabled. [ 0.000000] Rude variant of Tasks RCU enabled. [ 0.000000] Tracing variant of Tasks RCU enabled. [ 0.000000] rcu: RCU calculated value of scheduler-enlistment delay is 100 jiffies. [ 0.000000] rcu: Adjusting geometry for rcu_fanout_leaf=16, nr_cpu_ids=4 [ 0.000000] NR_IRQS: 524544, nr_irqs: 456, preallocated irqs: 16 [ 0.000000] random: get_random_bytes called from start_kernel+0x622/0x9a8 with crng_init=0 [ 0.001000] Console: colour *CGA 80x25 [ 0.001000] printk: console [ttyS1] enabled [ 0.001000] ACPI: Core revision 20220331 [ 0.001000] clocksource: hpet: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 19112604467 ns [ 0.001015] APIC: Switch to symmetric I/O mode setup [ 0.003242] x2apic enabled [ 0.004012] Switched APIC routing to physical x2apic. [ 0.005020] kvm-guest: setup PV IPIs [ 0.007886] ..TIMER: vector=0x30 apic1=0 pin1=2 apic2=-1 pin2=-1 [ 0.008000] clocksource: tsc-early: mask: 0xffffffffffffffff max_cycles: 0x229835b7123, max_idle_ns: 440795242976 ns [ 0.008031] Calibrating delay loop (skipped) preset value.. 4799.99 BogoMIPS (lpj=2399998) [ 0.009017] pid_max: default: 32768 minimum: 301 [ 0.010169] LSM: Security Framework initializing [ 0.011083] Yama: becoming mindful. [ 0.012059] SELinux: Initializing. [ 0.013116] *** VALIDATE selinux *** [ 0.021636] Dentry cache hash table entries: 1048576 (order: 11, 8388608 bytes, vmalloc) [ 0.026751] Inode-cache hash table entries: 524288 (order: 10, 4194304 bytes, vmalloc) [ 0.028149] Mount-cache hash table entries: 16384 (order: 5, 131072 bytes, vmalloc) [ 0.029124] Mountpoint-cache hash table entries: 16384 (order: 5, 131072 bytes, vmalloc) [ 0.031091] *** VALIDATE tmpfs *** [ 0.033085] *** VALIDATE proc *** [ 0.034341] *** VALIDATE cgroup *** [ 0.035016] *** VALIDATE cgroup2 *** [ 0.037091] x86/cpu: User Mode Instruction Prevention (UMIP) activated [ 0.038181] Last level iTLB entries: 4KB 0, 2MB 0, 4MB 0 [ 0.039013] Last level dTLB entries: 4KB 0, 2MB 0, 4MB 0, 1GB 0 [ 0.040037] Spectre V2 : User space: Vulnerable [ 0.041013] Speculative Store Bypass: Vulnerable [ 0.044956] debug: unmapping init [mem 0xffffffff9ac59000-0xffffffff9ac60fff] [ 0.046839] smpboot: CPU0: Intel(R) Xeon(R) CPU E5-2695 v2 @ 2.40GHz (family: 0x6, model: 0x3e, stepping: 0x4) [ 0.047786] Performance Events: IvyBridge events, full-width counters, Intel PMU driver. [ 0.048030] ... version: 2 [ 0.049019] ... bit width: 48 [ 0.050020] ... generic registers: 4 [ 0.051020] ... value mask: 0000ffffffffffff [ 0.052022] ... max period: 00007fffffffffff [ 0.053020] ... fixed-purpose events: 3 [ 0.054019] ... event mask: 000000070000000f [ 0.056280] rcu: Hierarchical SRCU implementation. [ 0.058670] smp: Bringing up secondary CPUs ... [ 0.059780] x86: Booting SMP configuration: [ 0.060036] .... node #0, CPUs: #1 #2 #3 [ 0.063773] smp: Brought up 1 node, 4 CPUs [ 0.065017] smpboot: Max logical packages: 1 [ 0.066023] smpboot: Total of 4 processors activated (19199.98 BogoMIPS) [ 0.216030] node 0 deferred pages initialised in 148ms [ 0.219129] devtmpfs: initialized [ 0.220306] x86/mm: Memory block size: 128MB [ 0.224025] gcov: version magic: 0x41383552 [ 0.226335] clocksource: jiffies: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 1911260446275000 ns [ 0.231139] futex hash table entries: 1024 (order: 4, 65536 bytes, vmalloc) [ 0.234319] pinctrl core: initialized pinctrl subsystem [ 0.235156] [ 0.235522] ************************************************************* [ 0.239012] ** NOTICE NOTICE NOTICE NOTICE NOTICE NOTICE NOTICE ** [ 0.241011] ** ** [ 0.244014] ** IOMMU DebugFS SUPPORT HAS BEEN ENABLED IN THIS KERNEL ** [ 0.246013] ** ** [ 0.249014] ** This means that this kernel is built to expose internal ** [ 0.251008] ** IOMMU data structures, which may compromise security on ** [ 0.253014] ** your system. ** [ 0.256014] ** ** [ 0.258014] ** If you see this message and you are not debugging the ** [ 0.260010] ** kernel, report this immediately to your vendor! ** [ 0.261009] ** ** [ 0.266016] ** NOTICE NOTICE NOTICE NOTICE NOTICE NOTICE NOTICE ** [ 0.268013] ************************************************************* [ 0.271654] NET: Registered protocol family 16 [ 0.273430] DMA: preallocated 512 KiB GFP_KERNEL pool for atomic allocations [ 0.276059] DMA: preallocated 512 KiB GFP_KERNEL|GFP_DMA pool for atomic allocations [ 0.279061] DMA: preallocated 512 KiB GFP_KERNEL|GFP_DMA32 pool for atomic allocations [ 0.283010] cpuidle: using governor menu [ 0.284541] acpiphp: ACPI Hot Plug PCI Controller Driver version: 0.5 [ 0.287640] PCI: Using configuration type 1 for base access [ 0.289123] core: PMU erratum BJ122, BV98, HSD29 worked around, HT is on [ 0.296138] HugeTLB registered 1.00 GiB page size, pre-allocated 0 pages [ 0.298029] HugeTLB registered 2.00 MiB page size, pre-allocated 0 pages [ 0.302058] cryptd: max_cpu_qlen set to 1000 [ 0.304255] ACPI: Added _OSI(Module Device) [ 0.305026] ACPI: Added _OSI(Processor Device) [ 0.306011] ACPI: Added _OSI(3.0 _SCP Extensions) [ 0.308015] ACPI: Added _OSI(Processor Aggregator Device) [ 0.314943] ACPI: 1 ACPI AML tables successfully acquired and loaded [ 0.320467] ACPI: Interpreter enabled [ 0.322055] ACPI: PM: (supports S0 S3 S4 S5) [ 0.324014] ACPI: Using IOAPIC for interrupt routing [ 0.326101] PCI: Using host bridge windows from ACPI; if necessary, use "pci=nocrs" and report a bug [ 0.330395] ACPI: Enabled 2 GPEs in block 00 to 0F [ 0.340730] ACPI: PCI Root Bridge [PCI0] (domain 0000 [bus 00-ff]) [ 0.343042] acpi PNP0A03:00: _OSC: OS supports [ASPM ClockPM Segments MSI HPX-Type3] [ 0.345020] acpi PNP0A03:00: _OSC: not requesting OS control; OS requires [ExtendedConfig ASPM ClockPM MSI] [ 0.349089] acpi PNP0A03:00: fail to add MMCONFIG information, can't access extended PCI configuration space under this bridge. [ 0.354335] acpiphp: Slot [2] registered [ 0.356309] acpiphp: Slot [5] registered [ 0.358398] acpiphp: Slot [6] registered [ 0.361116] acpiphp: Slot [7] registered [ 0.362239] acpiphp: Slot [8] registered [ 0.364083] acpiphp: Slot [9] registered [ 0.366131] acpiphp: Slot [10] registered [ 0.367126] acpiphp: Slot [3] registered [ 0.369087] acpiphp: Slot [4] registered [ 0.370120] acpiphp: Slot [11] registered [ 0.372096] acpiphp: Slot [12] registered [ 0.374142] acpiphp: Slot [13] registered [ 0.375122] acpiphp: Slot [14] registered [ 0.377104] acpiphp: Slot [15] registered [ 0.379098] acpiphp: Slot [16] registered [ 0.380096] acpiphp: Slot [17] registered [ 0.382097] acpiphp: Slot [18] registered [ 0.384099] acpiphp: Slot [19] registered [ 0.385113] acpiphp: Slot [20] registered [ 0.387119] acpiphp: Slot [21] registered [ 0.389099] acpiphp: Slot [22] registered [ 0.390097] acpiphp: Slot [23] registered [ 0.392077] acpiphp: Slot [24] registered [ 0.394104] acpiphp: Slot [25] registered [ 0.396137] acpiphp: Slot [26] registered [ 0.398171] acpiphp: Slot [27] registered [ 0.400116] acpiphp: Slot [28] registered [ 0.402187] acpiphp: Slot [29] registered [ 0.404098] acpiphp: Slot [30] registered [ 0.406124] acpiphp: Slot [31] registered [ 0.408057] PCI host bridge to bus 0000:00 [ 0.410018] pci_bus 0000:00: root bus resource [io 0x0000-0x0cf7 window] [ 0.413025] pci_bus 0000:00: root bus resource [io 0x0d00-0xffff window] [ 0.416087] pci_bus 0000:00: root bus resource [mem 0x000a0000-0x000bffff window] [ 0.419025] pci_bus 0000:00: root bus resource [mem 0xc0000000-0xfebfffff window] [ 0.422027] pci_bus 0000:00: root bus resource [mem 0xe0000000000-0xe007fffffff window] [ 0.424029] pci_bus 0000:00: root bus resource [bus 00-ff] [ 0.427166] pci 0000:00:00.0: [8086:1237] type 00 class 0x060000 [ 0.431010] pci 0000:00:01.0: [8086:7000] type 00 class 0x060100 [ 0.433060] pci 0000:00:01.1: [8086:7010] type 00 class 0x010180 [ 0.443014] pci 0000:00:01.1: reg 0x20: [io 0xc320-0xc32f] [ 0.448900] pci 0000:00:01.1: legacy IDE quirk: reg 0x10: [io 0x01f0-0x01f7] [ 0.452017] pci 0000:00:01.1: legacy IDE quirk: reg 0x14: [io 0x03f6] [ 0.454022] pci 0000:00:01.1: legacy IDE quirk: reg 0x18: [io 0x0170-0x0177] [ 0.456029] pci 0000:00:01.1: legacy IDE quirk: reg 0x1c: [io 0x0376] [ 0.459581] pci 0000:00:01.3: [8086:7113] type 00 class 0x068000 [ 0.462933] pci 0000:00:01.3: quirk: [io 0x0600-0x063f] claimed by PIIX4 ACPI [ 0.466048] pci 0000:00:01.3: quirk: [io 0x0700-0x070f] claimed by PIIX4 SMB [ 0.469819] pci 0000:00:02.0: [1af4:1000] type 00 class 0x020000 [ 0.476016] pci 0000:00:02.0: reg 0x10: [io 0xc300-0xc31f] [ 0.492015] pci 0000:00:02.0: reg 0x20: [mem 0xe0000000000-0xe0000003fff 64bit pref] [ 0.498016] pci 0000:00:02.0: reg 0x30: [mem 0xfeb80000-0xfebbffff pref] [ 0.505130] pci 0000:00:05.0: [1af4:1001] type 00 class 0x010000 [ 0.515014] pci 0000:00:05.0: reg 0x10: [io 0xc000-0xc07f] [ 0.523027] pci 0000:00:05.0: reg 0x14: [mem 0xfebc0000-0xfebc0fff] [ 0.541015] pci 0000:00:05.0: reg 0x20: [mem 0xe0000004000-0xe0000007fff 64bit pref] [ 0.551090] pci 0000:00:06.0: [1af4:1001] type 00 class 0x010000 [ 0.559013] pci 0000:00:06.0: reg 0x10: [io 0xc080-0xc0ff] [ 0.566014] pci 0000:00:06.0: reg 0x14: [mem 0xfebc1000-0xfebc1fff] [ 0.582017] pci 0000:00:06.0: reg 0x20: [mem 0xe0000008000-0xe000000bfff 64bit pref] [ 0.591075] pci 0000:00:07.0: [1af4:1001] type 00 class 0x010000 [ 0.597015] pci 0000:00:07.0: reg 0x10: [io 0xc100-0xc17f] [ 0.603019] pci 0000:00:07.0: reg 0x14: [mem 0xfebc2000-0xfebc2fff] [ 0.627016] pci 0000:00:07.0: reg 0x20: [mem 0xe000000c000-0xe000000ffff 64bit pref] [ 0.639186] pci 0000:00:08.0: [1af4:1001] type 00 class 0x010000 [ 0.651015] pci 0000:00:08.0: reg 0x10: [io 0xc180-0xc1ff] [ 0.659017] pci 0000:00:08.0: reg 0x14: [mem 0xfebc3000-0xfebc3fff] [ 0.682027] pci 0000:00:08.0: reg 0x20: [mem 0xe0000010000-0xe0000013fff 64bit pref] [ 0.696025] pci 0000:00:09.0: [1af4:1001] type 00 class 0x010000 [ 0.712017] pci 0000:00:09.0: reg 0x10: [io 0xc200-0xc27f] [ 0.722022] pci 0000:00:09.0: reg 0x14: [mem 0xfebc4000-0xfebc4fff] [ 0.738017] pci 0000:00:09.0: reg 0x20: [mem 0xe0000014000-0xe0000017fff 64bit pref] [ 0.751476] pci 0000:00:0a.0: [1af4:1001] type 00 class 0x010000 [ 0.762024] pci 0000:00:0a.0: reg 0x10: [io 0xc280-0xc2ff] [ 0.776022] pci 0000:00:0a.0: reg 0x14: [mem 0xfebc5000-0xfebc5fff] [ 0.798025] pci 0000:00:0a.0: reg 0x20: [mem 0xe0000018000-0xe000001bfff 64bit pref] [ 0.815077] ACPI: PCI: Interrupt link LNKA configured for IRQ 10 [ 0.818459] ACPI: PCI: Interrupt link LNKB configured for IRQ 10 [ 0.822415] ACPI: PCI: Interrupt link LNKC configured for IRQ 11 [ 0.825380] ACPI: PCI: Interrupt link LNKD configured for IRQ 11 [ 0.828223] ACPI: PCI: Interrupt link LNKS configured for IRQ 9 [ 0.833037] iommu: Default domain type: Passthrough [ 0.835492] SCSI subsystem initialized [ 0.837186] ACPI: bus type USB registered [ 0.838097] usbcore: registered new interface driver usbfs [ 0.841097] usbcore: registered new interface driver hub [ 0.842085] usbcore: registered new device driver usb [ 0.844158] pps_core: LinuxPPS API ver. 1 registered [ 0.846060] pps_core: Software ver. 5.3.6 - Copyright 2005-2007 Rodolfo Giometti [ 0.849061] PTP clock support registered [ 0.851074] EDAC MC: Ver: 3.0.0 [ 0.853060] PCI: Using ACPI for IRQ routing [ 0.855610] NetLabel: Initializing [ 0.857016] NetLabel: domain hash size = 128 [ 0.858012] NetLabel: protocols = UNLABELED CIPSOv4 CALIPSO [ 0.860103] NetLabel: unlabeled traffic allowed by default [ 0.863188] vgaarb: loaded [ 0.865528] hpet0: at MMIO 0xfed00000, IRQs 2, 8, 0 [ 0.868097] hpet0: 3 comparators, 64-bit 100.000000 MHz counter [ 0.874466] clocksource: Switched to clocksource kvm-clock [ 0.985162] VFS: Disk quotas dquot_6.6.0 [ 0.987846] VFS: Dquot-cache hash table entries: 512 (order 0, 4096 bytes) [ 0.990966] *** VALIDATE ramfs *** [ 0.992222] *** VALIDATE hugetlbfs *** [ 0.994049] pnp: PnP ACPI init [ 0.996706] pnp: PnP ACPI: found 6 devices [ 1.015245] clocksource: acpi_pm: mask: 0xffffff max_cycles: 0xffffff, max_idle_ns: 2085701024 ns [ 1.019761] pci_bus 0000:00: resource 4 [io 0x0000-0x0cf7 window] [ 1.022338] pci_bus 0000:00: resource 5 [io 0x0d00-0xffff window] [ 1.024894] pci_bus 0000:00: resource 6 [mem 0x000a0000-0x000bffff window] [ 1.027763] pci_bus 0000:00: resource 7 [mem 0xc0000000-0xfebfffff window] [ 1.030873] pci_bus 0000:00: resource 8 [mem 0xe0000000000-0xe007fffffff window] [ 1.034341] NET: Registered protocol family 2 [ 1.037525] IP idents hash table entries: 131072 (order: 8, 1048576 bytes, vmalloc) [ 1.043615] tcp_listen_portaddr_hash hash table entries: 4096 (order: 5, 163840 bytes, vmalloc) [ 1.047572] TCP established hash table entries: 65536 (order: 7, 524288 bytes, vmalloc) [ 1.053493] TCP bind hash table entries: 65536 (order: 9, 2097152 bytes, vmalloc) [ 1.057524] TCP: Hash tables configured (established 65536 bind 65536) [ 1.061118] MPTCP token hash table entries: 8192 (order: 6, 393216 bytes, vmalloc) [ 1.065127] UDP hash table entries: 4096 (order: 6, 393216 bytes, vmalloc) [ 1.068326] UDP-Lite hash table entries: 4096 (order: 6, 393216 bytes, vmalloc) [ 1.071827] NET: Registered protocol family 1 [ 1.075041] RPC: Registered named UNIX socket transport module. [ 1.077507] RPC: Registered udp transport module. [ 1.079405] RPC: Registered tcp transport module. [ 1.081051] RPC: Registered tcp NFSv4.1 backchannel transport module. [ 1.083886] NET: Registered protocol family 44 [ 1.085733] pci 0000:00:00.0: Limiting direct PCI/PCI transfers [ 1.088065] pci 0000:00:01.0: PIIX3: Enabling Passive Release [ 1.090187] pci 0000:00:01.0: Activating ISA DMA hang workarounds [ 1.092560] PCI: CLS 0 bytes, default 64 [ 1.095123] Unpacking initramfs... [ 2.506239] debug: unmapping init [mem 0xffff9b97bcc54000-0xffff9b97bffbffff] [ 2.514748] PCI-DMA: Using software bounce buffering for IO (SWIOTLB) [ 2.517100] software IO TLB: mapped [mem 0x00000000a8000000-0x00000000ac000000] (64MB) [ 2.521665] clocksource: tsc: mask: 0xffffffffffffffff max_cycles: 0x229835b7123, max_idle_ns: 440795242976 ns [ 3.035576] Initialise system trusted keyrings [ 3.037108] Key type blacklist registered [ 3.039555] workingset: timestamp_bits=36 max_order=20 bucket_order=0 [ 3.049123] zbud: loaded [ 3.052485] *** VALIDATE nfs *** [ 3.053654] *** VALIDATE nfs4 *** [ 3.055402] pstore: using deflate compression [ 3.059879] Platform Keyring initialized [ 3.179745] NET: Registered protocol family 38 [ 3.181168] Key type asymmetric registered [ 3.183301] Asymmetric key parser 'x509' registered [ 3.187794] Block layer SCSI generic (bsg) driver version 0.4 loaded (major 247) [ 3.191199] io scheduler mq-deadline registered [ 3.193094] io scheduler kyber registered [ 3.194602] io scheduler bfq registered [ 3.196708] atomic64_test: passed for x86-64 platform with CX8 and with SSE [ 3.200230] shpchp: Standard Hot Plug PCI Controller Driver version: 0.4 [ 3.203266] input: Power Button as /devices/LNXSYSTM:00/LNXPWRBN:00/input/input0 [ 3.206317] ACPI: Power Button [PWRF] [ 3.211993] ACPI: \_SB_.LNKB: Enabled at IRQ 10 [ 3.219230] ACPI: \_SB_.LNKA: Enabled at IRQ 11 [ 3.234042] ACPI: \_SB_.LNKC: Enabled at IRQ 11 [ 3.241281] ACPI: \_SB_.LNKD: Enabled at IRQ 10 [ 3.261917] Serial: 8250/16550 driver, 4 ports, IRQ sharing enabled [ 3.290297] 00:03: ttyS1 at I/O 0x2f8 (irq = 3, base_baud = 115200) is a 16550A [ 3.318241] 00:04: ttyS0 at I/O 0x3f8 (irq = 4, base_baud = 115200) is a 16550A [ 3.326473] Non-volatile memory driver v1.3 [ 3.328301] Linux agpgart interface v0.103 [ 3.358676] virtio_blk virtio1: [vda] 146960 512-byte logical blocks (75.2 MB/71.8 MiB) [ 3.361421] vda: detected capacity change from 0 to 75243520 [ 3.374554] virtio_blk virtio2: [vdb] 2097152 512-byte logical blocks (1.07 GB/1.00 GiB) [ 3.377190] vdb: detected capacity change from 0 to 1073741824 [ 3.392607] virtio_blk virtio3: [vdc] 5120000 512-byte logical blocks (2.62 GB/2.44 GiB) [ 3.394846] vdc: detected capacity change from 0 to 2621440000 [ 3.409803] virtio_blk virtio4: [vdd] 5120000 512-byte logical blocks (2.62 GB/2.44 GiB) [ 3.412098] vdd: detected capacity change from 0 to 2621440000 [ 3.427829] virtio_blk virtio5: [vde] 8388608 512-byte logical blocks (4.29 GB/4.00 GiB) [ 3.430955] vde: detected capacity change from 0 to 4294967296 [ 3.446516] virtio_blk virtio6: [vdf] 8388608 512-byte logical blocks (4.29 GB/4.00 GiB) [ 3.449982] vdf: detected capacity change from 0 to 4294967296 [ 3.458100] libphy: Fixed MDIO Bus: probed [ 3.465354] usbcore: registered new interface driver usbserial_generic [ 3.467683] usbserial: USB Serial support registered for generic [ 3.470119] i8042: PNP: PS/2 Controller [PNP0303:KBD,PNP0f13:MOU] at 0x60,0x64 irq 1,12 [ 3.473984] serio: i8042 KBD port at 0x60,0x64 irq 1 [ 3.475532] serio: i8042 AUX port at 0x60,0x64 irq 12 [ 3.477694] mousedev: PS/2 mouse device common for all mice [ 3.480620] rtc_cmos 00:05: RTC can wake from S4 [ 3.483542] rtc_cmos 00:05: registered as rtc0 [ 3.484038] input: AT Translated Set 2 keyboard as /devices/platform/i8042/serio0/input/input1 [ 3.485472] rtc_cmos 00:05: alarms up to one day, y3k, 242 bytes nvram, hpet irqs [ 3.491613] input: VirtualPS/2 VMware VMMouse as /devices/platform/i8042/serio1/input/input4 [ 3.492176] intel_pstate: CPU model not supported [ 3.497231] input: VirtualPS/2 VMware VMMouse as /devices/platform/i8042/serio1/input/input3 [ 3.502028] hid: raw HID events driver (C) Jiri Kosina [ 3.504288] usbcore: registered new interface driver usbhid [ 3.506602] usbhid: USB HID core driver [ 3.508217] drop_monitor: Initializing network drop monitor service [ 3.510898] Initializing XFRM netlink socket [ 3.512913] NET: Registered protocol family 10 [ 3.515928] Segment Routing with IPv6 [ 3.517682] NET: Registered protocol family 17 [ 3.520137] mpls_gso: MPLS GSO support [ 3.525252] RAS: Correctable Errors collector initialized. [ 3.527049] AVX version of gcm_enc/dec engaged. [ 3.528400] AES CTR mode by8 optimization enabled [ 3.596060] sched_clock: Marking stable (3596032852, 0)->(4486125938, -890093086) [ 3.599739] registered taskstats version 1 [ 3.602399] Loading compiled-in X.509 certificates [ 3.604644] zswap: loaded using pool lzo/zbud [ 3.628268] Key type big_key registered [ 3.639499] Key type encrypted registered [ 3.640712] ima: No TPM chip found, activating TPM-bypass! [ 3.642285] ima: Allocated hash algorithm: sha1 [ 3.643782] ima: No architecture policies found [ 3.645453] evm: Initialising EVM extended attributes: [ 3.647276] evm: security.selinux [ 3.648487] evm: security.ima [ 3.649319] evm: security.capability [ 3.650468] evm: HMAC attrs: 0x1 [ 3.653207] rtc_cmos 00:05: setting system clock to 2026-08-27 12:40:51 UTC (1787834451) [ 3.659547] debug: unmapping init [mem 0xffffffff9bc03000-0xffffffff9bdfffff] [ 3.662472] debug: unmapping init [mem 0xffffffff9a982000-0xffffffff9ac58fff] [ 3.670271] Write protecting the kernel read-only data: 28672k [ 3.674177] debug: unmapping init [mem 0xffffffff99003000-0xffffffff991fffff] [ 3.677631] debug: unmapping init [mem 0xffffffff99914000-0xffffffff999fffff] [ 3.712573] systemd[1]: systemd 239 (239-82.el8_10.5) running in system mode. (+PAM +AUDIT +SELINUX +IMA -APPARMOR +SMACK +SYSVINIT +UTMP +LIBCRYPTSETUP +GCRYPT +GNUTLS +ACL +XZ +LZ4 +SECCOMP +BLKID +ELFUTILS +KMOD +IDN2 -IDN +PCRE2 default-hierarchy=legacy) [ 3.722667] systemd[1]: Detected virtualization kvm. [ 3.724625] systemd[1]: Detected architecture x86-64. [ 3.726416] systemd[1]: Running in initial RAM disk. Welcome to Rocky Linux 8.10 (Green Obsidian) dracut-049-233.git20240115.el8 (Initramfs)! [ 3.756394] systemd[1]: No hostname configured. [ 3.757847] systemd[1]: Set hostname to . [ 3.759739] random: systemd: uninitialized urandom read (16 bytes read) [ 3.762132] systemd[1]: Initializing machine ID from random generator. [ 3.889843] random: systemd: uninitialized urandom read (16 bytes read) [ 3.899718] systemd[1]: Reached target Slices. [ OK ] Reached target Slices. [ 3.906203] random: systemd: uninitialized urandom read (16 bytes read) [ 3.909247] systemd[1]: Reached target Local File Systems. [ OK ] Reached target Local File Systems. [ 3.914806] systemd[1]: Reached target Swap. [ OK ] Reached target Swap. [ OK ] Listening on udev Kernel Socket. [ OK ] Started Dispatch Password Requests to Console Directory Watch. [ OK ] Reached target Local Encrypted Volumes. [ OK ] Listening on Journal Socket (/dev/log). [ OK ] Reached target Timers. [ OK ] Reached target Paths. [ OK ] Listening on udev Control Socket. [ OK ] Listening on Journal Socket. Starting Create Volatile Files and Directories... Starting Journal Service... Starting Apply Kernel Variables... [ OK ] Reached target Sockets. Starting Setup Virtual Console... [ OK ] Started Memstrack Anylazing Service. Starting Create list of required st…ce nodes for the current kernel... [ OK ] Reached target Initrd Root Device. [ OK ] Started Create Volatile Files and Directories. [ OK ] Started Apply Kernel Variables. [ OK ] Started Setup Virtual Console. [ OK ] Started Create list of required sta…vice nodes for the current kernel. Starting Create Static Device Nodes in /dev... Starting dracut cmdline hook... [ OK ] Started Create Static Device Nodes in /dev. [ OK ] Started Journal Service. [ OK ] Started dracut cmdline hook. Starting dracut pre-udev hook... [ 4.603125] device-mapper: uevent: version 1.0.3 [ 4.605104] device-mapper: ioctl: 4.46.0-ioctl (2022-02-22) initialised: dm-devel@redhat.com [ OK ] Started dracut pre-udev hook. Starting udev Kernel Device Manager... [ OK ] Started udev Kernel Device Manager. Starting dracut pre-trigger hook... [ OK ] Started dracut pre-trigger hook. Starting udev Coldplug all Devices... Mounting Kernel Configuration File System... [ OK ] Mounted Kernel Configuration File System. [ OK ] Started udev Coldplug all Devices. Starting dracut initqueue hook... [ OK ] Reached target System Initialization. [ OK ] Reached target Basic System. [ OK ] Started Hardware RNG Entropy Gatherer Daemon. [ 5.321301] virtio_net virtio0 ens2: renamed from eth0 [ 5.336937] random: fast init done [ 5.402707] scsi host0: ata_piix [ 5.421221] scsi host1: ata_piix [ 5.422964] ata1: PATA max MWDMA2 cmd 0x1f0 ctl 0x3f6 bmdma 0xc320 irq 14 [ 5.425371] ata2: PATA max MWDMA2 cmd 0x170 ctl 0x376 bmdma 0xc328 irq 15 [ 9.733356] dracut-initqueue[584]: RTNETLINK answers: File exists [ 10.156100] random: crng init done [ 10.158505] random: 7 urandom warning(s) missed due to ratelimiting Starting nbd nbd0... [ OK ] Started nbd nbd0. [ OK ] Started dracut initqueue hook. Mounting /sysroot... [ OK ] Reached target Remote File Systems (Pre). [ OK ] Reached target Remote File Systems. [ 10.593389] EXT4-fs (nbd0): mounted filesystem with ordered data mode. Opts: (null) [ OK ] Mounted /sysroot. [ OK ] Reached target Initrd Root File System. Starting Reload Configuration from the Real Root... [ OK ] Started Reload Configuration from the Real Root. [ OK ] Reached target Initrd File Systems. [ OK ] Reached target Initrd Default Target. Starting dracut pre-pivot and cleanup hook... [ OK ] Started dracut pre-pivot and cleanup hook. Starting Cleaning Up and Shutting Down Daemons... Stopping Hardware RNG Entropy Gatherer Daemon... [ OK ] Stopped dracut pre-pivot and cleanup hook. [ OK ] Stopped target Initrd Default Target. [ OK ] Stopped target Remote File Systems. [ OK ] Stopped target Timers. [ OK ] Stopped target Initrd Root Device. [ OK ] Stopped target Remote File Systems (Pre). [ OK ] Stopped dracut initqueue hook. [ OK ] Stopped Hardware RNG Entropy Gatherer Daemon. [ OK ] Stopped target Basic System. [ OK ] Stopped target System Initialization. [ OK ] Stopped udev Coldplug all Devices. [ OK ] Stopped dracut pre-trigger hook. Stopping udev Kernel Device Manager... [ OK ] Stopped target Swap. [ OK ] Stopped Apply Kernel Variables. [ OK ] Stopped Create Volatile Files and Directories. [ OK ] Stopped target Local File Systems. [ OK ] Stopped target Local Encrypted Volumes. [ OK ] Stopped target Sockets. [ OK ] Stopped target Slices. [ OK ] Stopped target Paths. [ OK ] Stopped Dispatch Password Requests to Console Directory Watch. [ OK ] Stopped udev Kernel Device Manager. [ OK ] Started Cleaning Up and Shutting Down Daemons. [ OK ] Stopped Create Static Device Nodes in /dev. [ OK ] Stopped Create list of required sta…vice nodes for the current kernel. [ OK ] Stopped dracut pre-udev hook. [ OK ] Stopped dracut cmdline hook. [ OK ] Closed udev Kernel Socket. [ OK ] Closed udev Control Socket. Starting Cleanup udevd DB... [ OK ] Started Cleanup udevd DB. [ OK ] Reached target Switch Root. Starting Switch Root... [ 11.737269] printk: systemd: 25 output lines suppressed due to ratelimiting [ 12.001222] SELinux: Disabled at runtime. [ 12.064291] systemd[1]: systemd 239 (239-82.el8_10.5) running in system mode. (+PAM +AUDIT +SELINUX +IMA -APPARMOR +SMACK +SYSVINIT +UTMP +LIBCRYPTSETUP +GCRYPT +GNUTLS +ACL +XZ +LZ4 +SECCOMP +BLKID +ELFUTILS +KMOD +IDN2 -IDN +PCRE2 default-hierarchy=legacy) [ 12.073704] systemd[1]: Detected virtualization kvm. [ 12.076015] systemd[1]: Detected architecture x86-64. Welcome to Rocky Linux 8.10 (Green Obsidian)! [ 12.576873] systemd[1]: initrd-switch-root.service: Succeeded. [ 12.580547] systemd[1]: Stopped Switch Root. [ OK ] Stopped Switch Root. [ 12.585367] systemd[1]: systemd-journald.service: Service has no hold-off time (RestartSec=0), scheduling restart. [ 12.589279] systemd[1]: systemd-journald.service: Scheduled restart job, restart counter is at 1. [ 12.592773] systemd[1]: Stopped Journal Service. [ OK ] Stopped Journal Service. [ 12.603116] systemd[1]: Starting Journal Service... Starting Journal Service... [ 12.610689] systemd[1]: Listening on udev Control Socket. [ OK ] Listening on udev Control Socket. [ OK ] Created slice system-sshd\x2dkeygen.slice. [ OK ] Created slice User and Session Slice. [ OK ] Listening on RPCbind Server Activation Socket. [ OK ] Created slice system-getty.slice. [ OK ] Reached target RPC Port Mapper. [ OK ] Reached target Slices. Starting Create list of required st…ce nodes for the current kernel... Activating swap /dev/disk/by-label/SWAP... [ OK ] Started Forward Password Requests to Wall Directory Watch. [ OK ] Listening on initctl Compatibility Named Pipe. [FAILED] Failed to set up automount Arbitrar…rmats File System Automount Point. See 'systemctl status proc-sys-fs-binfmt_misc.automount' for details. [ OK ] Reached target rpc_pipefs.target. [ OK ] Stopped target Switch Root. [ OK ] Stopped target Initrd File Systems. [ OK ] Stopped target Initrd Root File System. [ OK ] Created slice system-serial\x2dgetty.slice. [ OK ] Listening on udev Kernel Socket. [ 12.703945] Adding 1048572k swap on /dev/vdb. Priority:-2 extents:1 across:1048572k FS Starting udev Coldplug all Devices... Starting Remount Root and Kernel File Systems... [ OK ] Started Dispatch Password Requests to Console Directory Watch. [ OK ] Reached target Local Encrypted Volumes. Mounting Huge Pages File System... [ OK ] Reached target Paths. Mounting Kernel Debug File System... [ OK ] Listening on Process Core Dump Socket. Mounting POSIX Message Queue File System... Starting Apply Kernel Variables... [ OK ] Started Journal Service. [ OK ] Started Create list of required sta…vice nodes for the current kernel. [ OK ] Activated swap /dev/disk/by-label/SWAP. [FAILED] Failed to start Remount Root and Kernel File Systems. See 'systemctl status systemd-remount-fs.service' for details. [ OK ] Mounted Huge Pages File System. [ OK ] Mounted Kernel Debug File System. [ OK ] Mounted POSIX Message Queue File System. [ OK ] Started Apply Kernel Variables. Starting Configure read-only root support... [ OK ] Reached target Swap. Starting Create Static Device Nodes in /dev... Starting Flush Journal to Persistent Storage... [ OK ] Started Flush Journal to Persistent Storage. [ OK ] Started Create Static Device Nodes in /dev. Starting udev Kernel Device Manager... [ OK ] Reached target Local File Systems (Pre). Mounting /home/green/git/lustre-release... Mounting /mnt... [ OK ] Started udev Coldplug all Devices. [ OK ] Mounted /mnt. [ 13.097749] squashfs: version 4.0 (2009/01/31) Phillip Lougher [ OK ] Mounted /home/green/git/lustre-release. [ OK ] Started udev Kernel Device Manager. [ 13.433163] input: PC Speaker as /devices/platform/pcspkr/input/input5 [ 13.457947] piix4_smbus 0000:00:01.3: SMBus Host Controller at 0x700, revision 0 [ 13.555962] RAPL PMU: API unit is 2^-32 Joules, 0 fixed counters, 10737418240 ms ovfl timer [ 13.565851] EDAC sbridge: Ver: 1.1.2 [ 15.264407] Key type dns_resolver registered [ 15.568962] NFS: Registering the id_resolver key type [ 15.571080] Key type id_resolver registered [ 15.572628] Key type id_legacy registered [ OK ] Started Configure read-only root support. [ OK ] Reached target Local File Systems. Starting Rebuild Dynamic Linker Cache... Starting Mark the need to relabel after reboot... Starting Create Volatile Files and Directories... Starting Load/Save Random Seed... [ OK ] Started Mark the need to relabel after reboot. [ OK ] Started Load/Save Random Seed. [ OK ] Started Create Volatile Files and Directories. Starting Update UTMP about System Boot/Shutdown... Starting RPC Bind... [ OK ] Started Update UTMP about System Boot/Shutdown. [ OK ] Started RPC Bind. [ OK ] Started Rebuild Dynamic Linker Cache. Starting Update is Completed... [ OK ] Started Update is Completed. [ OK ] Reached target System Initialization. [ OK ] Started Daily Cleanup of Temporary Directories. [ OK ] Started daily update of the root trust anchor for DNSSEC. [ OK ] Started dnf makecache --timer. [ OK ] Reached target Timers. [ OK ] Listening on D-Bus System Message Bus Socket. [ OK ] Reached target Sockets. [ OK ] Reached target Basic System. Starting Restore /run/initramfs on shutdown... [ OK ] Started Hardware RNG Entropy Gatherer Daemon. [ OK ] Reached target sshd-keygen.target. [ OK ] Started irqbalance daemon. [ OK ] Started D-Bus System Message Bus. Starting Network Manager... Starting Login Service... [ OK ] Started Restore /run/initramfs on shutdown. [ OK ] Started Login Service. [ OK ] Started Network Manager. [ OK ] Reached target Network. Starting Dynamic System Tuning Daemon... Starting OpenSSH server daemon... Starting GSSAPI Proxy Daemon... Starting Network Manager Wait Online... Starting Hostname Service... [ OK ] Started OpenSSH server daemon. [ OK ] Started GSSAPI Proxy Daemon. [ OK ] Reached target NFS client services. [ OK ] Reached target Remote File Systems (Pre). [ OK ] Reached target Remote File Systems. Starting Permit User Sessions... [ OK ] Started Permit User Sessions. [ OK ] Started Getty on tty1. [ OK ] Started Serial Getty on ttyS0. [ OK ] Started Command Scheduler. [ OK ] Started Serial Getty on ttyS1. [ OK ] Reached target Login Prompts. [ OK ] Started Hostname Service. Starting Network Manager Script Dispatcher Service... [ OK ] Started Network Manager Script Dispatcher Service. [ OK ] Started Network Manager Wait Online. [ OK ] Reached target Network is Online. Starting Notify NFS peers of a restart... Starting System Logging Service... Starting Crash recovery kernel arming... [ OK ] Started Notify NFS peers of a restart. [ OK ] Started System Logging Service. Starting Authorization Manager... [ OK ] Started Authorization Manager. [ OK ] Started Dynamic System Tuning Daemon. [ OK ] Reached target Multi-User System. [ OK ] Reached target Graphical Interface. Starting Update UTMP about System Runlevel Changes... [ OK ] Started Update UTMP about System Runlevel Changes. Rocky Linux 8.10 (Green Obsidian) Kernel 4.18.0rh8.10-debug on an x86_64 oleg619-server login: [ 62.072729] libcfs: loading out-of-tree module taints kernel. [ 62.144165] Key type ._llcrypt registered [ 62.147387] Key type .llcrypt registered [ 62.245569] Lustre: DEBUG MARKER: oleg619-server.virtnet: executing set_hostid [ 81.615584] Lustre: DEBUG MARKER: oleg619-server.virtnet: executing load_modules_local [ 82.911543] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 82.926287] alg: No test for adler32 (adler32-zlib) [ 84.415831] Lustre: Lustre: Build Version: 2.17.57_98_gaf12cc4 [ 85.026890] LNet: Added LNI 192.168.206.119@tcp [8/256/0/180] [ 86.736158] Key type lgssc registered [ 88.230879] Lustre: Echo OBD driver; http://www.lustre.org/ [ 103.448782] ZFS: Loaded module v2.3.2-1, ZFS pool version 5000, ZFS filesystem version 5 [ 141.436826] Lustre: DEBUG MARKER: oleg619-server.virtnet: executing load_modules_local [ 154.038255] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 154.085527] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 155.315963] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 155.346802] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 155.422841] Lustre: lustre-MDT0000: new disk, initializing [ 155.507884] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 155.520732] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 159.280438] Lustre: DEBUG MARKER: oleg619-server.virtnet: executing set_default_debug vfstrace rpctrace dlmtrace neterror ha config ioctl super lfsck all [ 172.001538] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 172.161631] Lustre: 6502:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 172.209856] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [ 172.212834] Lustre: Skipped 1 previous similar message [ 172.323417] Lustre: lustre-MDT0001: new disk, initializing [ 172.468076] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 172.514893] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [ 172.537556] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 177.115237] Lustre: DEBUG MARKER: oleg619-server.virtnet: executing set_default_debug vfstrace rpctrace dlmtrace neterror ha config ioctl super lfsck all [ 181.790497] Lustre: Modifying parameter general.debug_raw_pointers=Y in log params [ 191.336748] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 191.602394] Lustre: lustre-OST0000: new disk, initializing [ 191.609517] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 191.620949] Lustre: 8441:0:(osd_compat.c:1353:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 191.686467] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 196.659894] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [ 196.677206] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 196.798258] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 197.795092] Lustre: DEBUG MARKER: oleg619-server.virtnet: executing set_default_debug vfstrace rpctrace dlmtrace neterror ha config ioctl super lfsck all [ 213.193455] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 213.320096] Lustre: lustre-OST0001: new disk, initializing [ 213.326548] Lustre: srv-lustre-OST0001: No data found on store. Initialize space. [ 213.341534] Lustre: 9515:0:(osd_compat.c:1353:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 213.419847] Lustre: lustre-OST0001: Imperative Recovery not enabled, recovery window 60-180 [ 215.195068] hrtimer: interrupt took 12073268 ns [ 219.176748] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x00000002c0000400-0x0000000300000400]:1:ost [ 219.195117] Lustre: cli-lustre-OST0001-super: Allocated super-sequence [0x00000002c0000400-0x0000000300000400]:1:ost] [ 219.263433] Lustre: lustre-OST0001-osc-MDT0000: update sequence from 0x100010000 to 0x2c0000401 [ 219.528538] Lustre: DEBUG MARKER: oleg619-server.virtnet: executing set_default_debug vfstrace rpctrace dlmtrace neterror ha config ioctl super lfsck all [ 231.307507] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 241.995506] Lustre: Setting parameter general.lod.*.mdt_hash=crush in log params [ 248.558657] Lustre: DEBUG MARKER: oleg619-server.virtnet: executing check_logdir /tmp/testlogs/ [ 253.018543] Lustre: DEBUG MARKER: oleg619-server.virtnet: executing yml_node [ 256.454684] Lustre: DEBUG MARKER: Client: 2.17.57.98 [ 258.592181] Lustre: DEBUG MARKER: MDS: 2.17.57.98 [ 260.483713] Lustre: DEBUG MARKER: OSS: 2.17.57.98 [ 261.927733] Lustre: DEBUG MARKER: -----============= acceptance-small: conf-sanity ============----- Thu Aug 27 08:45:07 EDT 2026 [ 277.138976] Lustre: DEBUG MARKER: excepting tests: 102 106 115 32newtarball [ 278.574729] Lustre: DEBUG MARKER: skipping tests SLOW=no: 45 69 106 111 114 [ 295.392906] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 295.405265] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 295.443943] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 295.929776] Lustre: lustre-MDT0000-lwp-OST0000: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 295.950178] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 295.962581] Lustre: Skipped 2 previous similar messages [ 299.945467] Lustre: server umount lustre-MDT0000 complete [ 303.694179] LustreError: 6494:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1787834751 with bad export cookie 9746011922714108155 [ 303.701336] LustreError: MGC192.168.206.119@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 303.708728] LustreError: 6494:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 2 previous similar messages [ 306.147568] Lustre: lustre-MDT0001-lwp-OST0001: Connection to lustre-MDT0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 306.149391] LustreError: 6510:0:(ldlm_lib.c:1228:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 306.149531] Lustre: lustre-MDT0001: Not available for connect from 0@lo (stopping) [ 306.172846] Lustre: Skipped 3 previous similar messages [ 306.218743] LustreError: 6510:0:(ldlm_lib.c:1228:target_handle_connect()) Skipped 6 previous similar messages [ 308.193384] LustreError: 8392:0:(ldlm_lib.c:1228:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 308.193812] Lustre: lustre-MDT0001: Not available for connect from 0@lo (stopping) [ 308.247831] Lustre: Skipped 2 previous similar messages [ 310.247861] Lustre: server umount lustre-MDT0001 complete [ 314.750640] Lustre: server umount lustre-OST0000 complete [ 319.078937] Lustre: server umount lustre-OST0001 complete [ 326.324388] Lustre: DEBUG MARKER: oleg619-server.virtnet: executing set_hostid [ 335.158581] Lustre: DEBUG MARKER: oleg619-server.virtnet: executing load_modules_local [ 372.266783] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 372.579759] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 372.618618] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 372.719101] Lustre: lustre-MDT0000: new disk, initializing [ 372.833451] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 372.859481] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 378.821479] Lustre: DEBUG MARKER: oleg619-server.virtnet: executing set_default_debug -1 all [ 388.764393] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 388.879451] Lustre: 15061:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 388.970632] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [ 388.975162] Lustre: Skipped 1 previous similar message [ 389.056755] Lustre: lustre-MDT0001: new disk, initializing [ 389.132708] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 389.169224] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [ 389.190261] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 393.281545] Lustre: DEBUG MARKER: oleg619-server.virtnet: executing set_default_debug -1 all [ 402.360215] Lustre: DEBUG MARKER: oleg619-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 407.886456] Lustre: DEBUG MARKER: oleg619-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 413.952547] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 414.258888] Lustre: lustre-OST0000: new disk, initializing [ 414.264133] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 414.271873] Lustre: 17032:0:(osd_compat.c:1353:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 414.346254] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 415.345242] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [ 415.358727] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 415.452656] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 419.273765] Lustre: DEBUG MARKER: oleg619-server.virtnet: executing set_default_debug -1 all [ 428.184897] Lustre: DEBUG MARKER: oleg619-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 432.355710] Lustre: DEBUG MARKER: oleg619-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 [ 432.575132] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [ 436.671862] Lustre: DEBUG MARKER: oleg619-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid 50 [ 436.840791] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [ 441.320321] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 441.325599] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 441.335296] Lustre: Skipped 1 previous similar message [ 441.336961] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 441.343275] Lustre: Skipped 1 previous similar message [ 444.760317] Lustre: server umount lustre-OST0000 complete [ 450.532581] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 450.539091] LustreError: Skipped 1 previous similar message [ 450.548943] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 450.571822] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 462.304122] Lustre: lustre-MDT0000 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 462.437659] Lustre: server umount lustre-MDT0000 complete [ 465.339678] LustreError: 15056:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1787834913 with bad export cookie 9746011922714109688 [ 465.342151] LustreError: MGC192.168.206.119@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 465.349134] LustreError: 15056:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 1 previous similar message [ 465.643904] Lustre: server umount lustre-MDT0001 complete [ 469.606384] Lustre: DEBUG MARKER: == conf-sanity test 76a: set permanent params with lctl across mounts ========================================================== 08:48:35 (1787834915) [ 474.985877] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 475.317919] LustreError: 19295:0:(ldlm_lib.c:1228:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 475.349154] LustreError: 19295:0:(ldlm_lib.c:1228:target_handle_connect()) Skipped 1 previous similar message [ 475.399779] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 479.171283] Lustre: DEBUG MARKER: oleg619-server.virtnet: executing set_default_debug -1 all [ 480.739437] LustreError: 19295:0:(ldlm_lib.c:1228:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 485.731576] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 485.863468] Lustre: lustre-MDT0001: Not available for connect from 0@lo (not set up) [ 485.872781] Lustre: Skipped 6 previous similar messages [ 489.567114] Lustre: DEBUG MARKER: oleg619-server.virtnet: executing set_default_debug -1 all [ 495.864188] Lustre: DEBUG MARKER: oleg619-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 500.532882] Lustre: DEBUG MARKER: oleg619-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 506.140043] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 511.615150] Lustre: DEBUG MARKER: oleg619-server.virtnet: executing set_default_debug -1 all [ 517.672936] Lustre: DEBUG MARKER: oleg619-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 525.745812] Lustre: Modifying parameter general.osc.*.max_dirty_mb=452 in log params [ 531.937839] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 531.944146] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 531.953324] Lustre: Skipped 1 previous similar message [ 531.958932] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 537.775963] Lustre: server umount lustre-MDT0000 complete [ 541.158393] LustreError: 19296:0:(ldlm_lib.c:1228:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 541.183951] LustreError: 19296:0:(ldlm_lib.c:1228:target_handle_connect()) Skipped 2 previous similar messages [ 541.302185] LustreError: 20029:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1787834989 with bad export cookie 9746011922714110577 [ 541.306324] LustreError: MGC192.168.206.119@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 541.316061] LustreError: 20029:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 541.595605] Lustre: server umount lustre-MDT0001 complete [ 554.586375] Lustre: server umount lustre-OST0000 complete [ 576.708094] Lustre: DEBUG MARKER: oleg619-server.virtnet: executing load_modules_local [ 585.923619] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 586.264409] LustreError: 23675:0:(ldlm_lib.c:1228:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 586.334672] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 586.346269] Lustre: Skipped 2 previous similar messages [ 590.491374] Lustre: DEBUG MARKER: oleg619-server.virtnet: executing set_default_debug -1 all [ 598.168324] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 602.731757] Lustre: DEBUG MARKER: oleg619-server.virtnet: executing set_default_debug -1 all [ 605.410248] Lustre: Setting parameter general.debug_raw_pointers=Y in log params [ 611.260682] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 617.486473] Lustre: DEBUG MARKER: oleg619-server.virtnet: executing set_default_debug -1 all [ 626.011574] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 626.217330] Lustre: lustre-OST0001: new disk, initializing [ 626.220230] Lustre: srv-lustre-OST0001: No data found on store. Initialize space. [ 626.227956] Lustre: 25989:0:(osd_compat.c:1353:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 628.114862] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x00000002c0000400-0x0000000300000400]:1:ost [ 628.130485] Lustre: cli-lustre-OST0001-super: Allocated super-sequence [0x00000002c0000400-0x0000000300000400]:1:ost] [ 628.280481] Lustre: lustre-OST0001-osc-MDT0000: update sequence from 0x100010000 to 0x2c0000401 [ 632.422806] Lustre: DEBUG MARKER: oleg619-server.virtnet: executing set_default_debug -1 all [ 641.809512] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 644.585806] Lustre: Setting parameter general.lod.*.mdt_hash=crush in log params [ 658.933367] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 658.934689] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 658.956135] Lustre: Skipped 5 previous similar messages [ 658.979110] Lustre: Skipped 6 previous similar messages [ 661.871761] Lustre: server umount lustre-MDT0000 complete [ 664.058586] LustreError: 23676:0:(ldlm_lib.c:1228:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 664.092532] LustreError: 23676:0:(ldlm_lib.c:1228:target_handle_connect()) Skipped 6 previous similar messages [ 666.178282] LustreError: 23656:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1787835114 with bad export cookie 9746011922714111424 [ 666.191764] LustreError: MGC192.168.206.119@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 666.202993] LustreError: 23656:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 666.544428] Lustre: server umount lustre-MDT0001 complete [ 681.156684] Lustre: server umount lustre-OST0000 complete [ 695.283693] Lustre: server umount lustre-OST0001 complete [ 716.014659] Lustre: DEBUG MARKER: oleg619-server.virtnet: executing load_modules_local [ 726.283640] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 726.840454] LustreError: 29008:0:(ldlm_lib.c:1228:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 727.011019] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 727.026890] Lustre: Skipped 3 previous similar messages [ 731.629909] Lustre: DEBUG MARKER: oleg619-server.virtnet: executing set_default_debug -1 all [ 739.356495] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 743.857802] Lustre: DEBUG MARKER: oleg619-server.virtnet: executing set_default_debug -1 all [ 747.092939] Lustre: 30158:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify general/debug_raw_pointers=Y (mode = 0) failed: rc = -17 [ 753.339590] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 759.244330] Lustre: DEBUG MARKER: oleg619-server.virtnet: executing set_default_debug -1 all [ 766.515703] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 773.301711] Lustre: DEBUG MARKER: oleg619-server.virtnet: executing set_default_debug -1 all [ 780.873760] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 784.083814] Lustre: 32036:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify general/lod.*.mdt_hash=crush (mode = 0) failed: rc = -17 [ 791.009489] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 791.017826] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 791.029684] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 795.451248] Lustre: server umount lustre-MDT0000 complete [ 797.676088] LustreError: 29745:0:(ldlm_lib.c:1228:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 797.696520] LustreError: 29745:0:(ldlm_lib.c:1228:target_handle_connect()) Skipped 6 previous similar messages [ 798.752626] LustreError: 28989:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1787835246 with bad export cookie 9746011922714112719 [ 798.768438] LustreError: MGC192.168.206.119@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 798.772193] LustreError: 28989:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 799.149478] Lustre: server umount lustre-MDT0001 complete [ 813.109986] Lustre: server umount lustre-OST0000 complete [ 826.570711] Lustre: server umount lustre-OST0001 complete [ 833.389923] Lustre: DEBUG MARKER: == conf-sanity test 76b: verify params log setup correctly ========================================================== 08:54:38 (1787835278) [ 858.236298] Lustre: DEBUG MARKER: oleg619-server.virtnet: executing load_modules_local [ 867.680289] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 871.977606] Lustre: DEBUG MARKER: oleg619-server.virtnet: executing set_default_debug -1 all [ 878.514217] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 882.486654] Lustre: DEBUG MARKER: oleg619-server.virtnet: executing set_default_debug -1 all [ 885.132505] Lustre: 35985:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify general/debug_raw_pointers=Y (mode = 0) failed: rc = -17 [ 890.566854] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 896.409700] Lustre: DEBUG MARKER: oleg619-server.virtnet: executing set_default_debug -1 all [ 904.988792] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 914.437315] Lustre: DEBUG MARKER: oleg619-server.virtnet: executing set_default_debug -1 all [ 924.721301] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 929.602821] Lustre: 37865:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify general/lod.*.mdt_hash=crush (mode = 0) failed: rc = -17 [ 935.393077] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 935.400518] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 935.411021] Lustre: Skipped 3 previous similar messages [ 940.900206] Lustre: server umount lustre-MDT0000 complete [ 941.543482] LustreError: 35573:0:(ldlm_lib.c:1228:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 941.560703] LustreError: 35573:0:(ldlm_lib.c:1228:target_handle_connect()) Skipped 9 previous similar messages [ 944.099082] LustreError: 34819:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1787835391 with bad export cookie 9746011922714113825 [ 944.107510] LustreError: MGC192.168.206.119@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 944.109048] LustreError: 34819:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 944.318403] Lustre: server umount lustre-MDT0001 complete [ 957.965925] Lustre: server umount lustre-OST0000 complete [ 972.940212] Lustre: server umount lustre-OST0001 complete [ 984.090775] Lustre: DEBUG MARKER: == conf-sanity test 76c: verify changelog_mask is applied with lctl set_param -P ========================================================== 08:57:09 (1787835429) [ 999.197741] Lustre: DEBUG MARKER: oleg619-server.virtnet: executing load_modules_local [ 1008.822543] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1009.299766] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 1009.316567] Lustre: Skipped 7 previous similar messages [ 1013.701810] Lustre: DEBUG MARKER: oleg619-server.virtnet: executing set_default_debug -1 all [ 1021.534247] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1025.740886] Lustre: DEBUG MARKER: oleg619-server.virtnet: executing set_default_debug -1 all [ 1028.378394] Lustre: 41158:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify general/debug_raw_pointers=Y (mode = 0) failed: rc = -17 [ 1034.092374] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 1040.363420] Lustre: DEBUG MARKER: oleg619-server.virtnet: executing set_default_debug -1 all [ 1047.064349] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 1052.258059] Lustre: DEBUG MARKER: oleg619-server.virtnet: executing set_default_debug -1 all [ 1058.719726] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 1061.847493] Lustre: 43035:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify general/lod.*.mdt_hash=crush (mode = 0) failed: rc = -17 [ 1062.686884] Lustre: Setting parameter general.mdd.*.changelog_mask=-CLOSE in log params [ 1062.694835] Lustre: Skipped 1 previous similar message [ 1064.933120] Lustre: lustre-MDT0000: Not available for connect from 192.168.206.19@tcp (stopping) [ 1064.941211] Lustre: Skipped 9 previous similar messages [ 1068.004626] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 1068.023355] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 1068.042943] Lustre: Skipped 3 previous similar messages [ 1071.283187] Lustre: server umount lustre-MDT0000 complete [ 1074.747813] LustreError: 39991:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1787835522 with bad export cookie 9746011922714114931 [ 1074.750935] LustreError: MGC192.168.206.119@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 1074.762603] LustreError: 39991:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 1081.214845] Lustre: server umount lustre-MDT0001 complete [ 1089.335222] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1099.745224] LustreError: 3637:0:(client.c:1404:ptlrpc_import_delay_req()) @@@ invalidate in flight req@ffff9b9703d1cf00 x1874680382610176/t0(0) o250->MGC192.168.206.119@tcp@0@lo:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl Rpc:NQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 [ 1103.741462] Lustre: DEBUG MARKER: oleg619-server.virtnet: executing set_default_debug -1 all [ 1109.304487] Lustre: lustre-MDT0000-lwp-OST0000: Connection restored to 0@lo (at 0@lo) [ 1111.437501] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1116.392123] Lustre: DEBUG MARKER: oleg619-server.virtnet: executing set_default_debug -1 all [ 1119.216898] Lustre: lustre-MDT0001-lwp-OST0000: Connection restored to 0@lo (at 0@lo) [ 1119.229510] Lustre: Skipped 1 previous similar message [ 1124.023540] Lustre: DEBUG MARKER: oleg619-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 1125.775887] Lustre: DEBUG MARKER: mdc.lustre-MDT0000-mdc-*.mds_server_uuid in FULL state after 0 sec [ 1131.993545] Lustre: DEBUG MARKER: oleg619-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 1133.544737] Lustre: DEBUG MARKER: mdc.lustre-MDT0001-mdc-*.mds_server_uuid in FULL state after 0 sec [ 1144.938996] Lustre: server umount lustre-MDT0000 complete [ 1148.644988] LustreError: 39989:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1787835596 with bad export cookie 9746011922714116058 [ 1148.658282] LustreError: MGC192.168.206.119@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 1148.672822] LustreError: 39989:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 3 previous similar messages [ 1149.041883] Lustre: server umount lustre-MDT0001 complete [ 1163.244215] Lustre: server umount lustre-OST0000 complete [ 1165.281176] Lustre: 3638:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1787835597/real 1787835597] req@ffff9b9807b34b40 x1874680382645760/t0(0) o400->lustre-MDT0001-lwp-OST0001@0@lo:12/10 lens 224/224 e 0 to 1 dl 1787835613 ref 1 fl Rpc:XNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 [ 1167.150972] Lustre: server umount lustre-OST0001 complete [ 1175.202132] Lustre: DEBUG MARKER: == conf-sanity test 76d: verify llite.*.xattr_cache can be set by 'lctl set_param -P' correctly ========================================================== 09:00:20 (1787835620) [ 1192.558349] Lustre: DEBUG MARKER: oleg619-server.virtnet: executing load_modules_local [ 1203.423108] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1203.816655] LustreError: 47641:0:(ldlm_lib.c:1228:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 1203.837110] LustreError: 47641:0:(ldlm_lib.c:1228:target_handle_connect()) Skipped 20 previous similar messages [ 1208.147791] Lustre: DEBUG MARKER: oleg619-server.virtnet: executing set_default_debug -1 all [ 1217.633823] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1223.677579] Lustre: DEBUG MARKER: oleg619-server.virtnet: executing set_default_debug -1 all [ 1227.126579] Lustre: 48808:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify general/debug_raw_pointers=Y (mode = 0) failed: rc = -17 [ 1234.413520] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 1241.051630] Lustre: DEBUG MARKER: oleg619-server.virtnet: executing set_default_debug -1 all [ 1250.857971] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 1257.609591] Lustre: DEBUG MARKER: oleg619-server.virtnet: executing set_default_debug -1 all [ 1265.505286] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 1270.880458] Lustre: Setting parameter general.llite.*.xattr_cache=0 in log params [ 1284.583168] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 1289.916289] Lustre: server umount lustre-MDT0000 complete [ 1295.046815] LustreError: 47622:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1787835742 with bad export cookie 9746011922714116849 [ 1295.048240] LustreError: MGC192.168.206.119@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 1295.063465] LustreError: 47622:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 1295.295578] Lustre: server umount lustre-MDT0001 complete [ 1308.940225] Lustre: server umount lustre-OST0000 complete [ 1322.951521] Lustre: server umount lustre-OST0001 complete [ 1332.580871] Lustre: DEBUG MARKER: == conf-sanity test 77: comma-separated MGS NIDs and failover node NIDs ========================================================== 09:02:57 (1787835777) [ 1334.988146] Lustre: DEBUG MARKER: SKIP: conf-sanity test_77 mixed loopback and real device not working [ 1336.965591] Lustre: DEBUG MARKER: == conf-sanity test 78: run resize2fs on MDT and OST filesystems ========================================================== 09:03:02 (1787835782) [ 1349.309508] Lustre: DEBUG MARKER: oleg619-server.virtnet: executing set_hostid [ 1359.735359] Lustre: DEBUG MARKER: oleg619-server.virtnet: executing load_modules_local [ 1380.455561] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1380.665336] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 1380.709233] Lustre: lustre-MDT0000: new disk, initializing [ 1380.791759] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 1385.165048] Lustre: DEBUG MARKER: oleg619-server.virtnet: executing set_default_debug -1 all [ 1394.722139] Lustre: DEBUG MARKER: oleg619-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 1403.174420] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 1403.738332] Lustre: lustre-OST0000: new disk, initializing [ 1403.770879] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 1403.782213] Lustre: Skipped 1 previous similar message [ 1403.785540] Lustre: 54882:0:(osd_compat.c:1353:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 1405.025241] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:0:ost [ 1405.039911] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:0:ost] [ 1405.078184] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x240000400 [ 1410.796934] Lustre: DEBUG MARKER: oleg619-server.virtnet: executing set_default_debug -1 all [ 1421.298125] Lustre: DEBUG MARKER: oleg619-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 1423.614664] Lustre: DEBUG MARKER: create test files [ 1456.105383] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 1456.113719] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 1456.124344] Lustre: Skipped 14 previous similar messages [ 1459.406690] Lustre: server umount lustre-OST0000 complete [ 1463.094198] Lustre: server umount lustre-MDT0000 complete [ 1473.467885] Lustre: DEBUG MARKER: oleg619-server.virtnet: executing unload_modules_local [ 1476.330529] Key type lgssc unregistered [ 1476.604467] LNet: 56513:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 1476.609401] LNetError: 56513:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 1476.627280] LNet: Removed LNI 192.168.206.119@tcp [ 1477.583139] Key type .llcrypt unregistered [ 1477.589151] Key type ._llcrypt unregistered [ 1503.193159] Key type ._llcrypt registered [ 1503.196907] Key type .llcrypt registered [ 1503.311752] Lustre: DEBUG MARKER: oleg619-server.virtnet: executing load_modules_local [ 1504.334959] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 1504.362924] alg: No test for adler32 (adler32-zlib) [ 1505.427263] Lustre: Lustre: Build Version: 2.17.57_98_gaf12cc4 [ 1505.645422] LNet: Added LNI 192.168.206.119@tcp [8/256/0/180] [ 1507.297043] Key type lgssc registered [ 1508.289045] Lustre: Echo OBD driver; http://www.lustre.org/ [ 1515.437424] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 1515.472978] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1516.919281] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 1521.821734] Lustre: DEBUG MARKER: oleg619-server.virtnet: executing set_default_debug -1 all [ 1531.451467] Lustre: DEBUG MARKER: oleg619-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 1539.801511] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 1540.220329] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 1541.256756] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x240000400:102 to 0x240000400:129) [ 1541.485422] Lustre: 59289:0:(osd_internal.h:1460:osd_trans_exec_op()) lustre-OST0000: opcode 8: before 510 < left 544, rollback = 9 [ 1541.495024] Lustre: 59289:0:(osd_handler.c:2090:osd_trans_dump_creds()) create: 32/128/5, destroy: 0/0/0 [ 1541.504507] Lustre: 59289:0:(osd_handler.c:2097:osd_trans_dump_creds()) attr_set: 0/0/0, xattr_set: 0/0/0 [ 1541.522281] Lustre: 59289:0:(osd_handler.c:2107:osd_trans_dump_creds()) write: 2/18/1, punch: 0/0/0, quota 1/3/0 [ 1541.540978] Lustre: 59289:0:(osd_handler.c:2114:osd_trans_dump_creds()) insert: 32/544/0, delete: 0/0/0 [ 1541.560439] Lustre: 59289:0:(osd_handler.c:2121:osd_trans_dump_creds()) ref_add: 0/0/0, ref_del: 0/0/0 [ 1548.736347] Lustre: DEBUG MARKER: oleg619-server.virtnet: executing set_default_debug -1 all [ 1557.045743] Lustre: DEBUG MARKER: oleg619-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 1562.935595] Lustre: DEBUG MARKER: check files after expanding the MDT and OST filesystems [ 1569.277828] Lustre: DEBUG MARKER: create more files after expanding the MDT and OST filesystems [ 1603.043075] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 1603.066064] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 1606.575658] Lustre: server umount lustre-OST0000 complete [ 1610.477410] Lustre: server umount lustre-MDT0000 complete [ 1622.242933] Lustre: DEBUG MARKER: oleg619-server.virtnet: executing unload_modules_local [ 1625.285618] Key type lgssc unregistered [ 1625.550505] LNet: 60861:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 1625.561590] LNetError: 60861:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 1625.575723] LNet: Removed LNI 192.168.206.119@tcp [ 1626.386512] Key type .llcrypt unregistered [ 1626.389661] Key type ._llcrypt unregistered [ 1654.546396] Key type ._llcrypt registered [ 1654.548524] Key type .llcrypt registered [ 1654.618587] Lustre: DEBUG MARKER: oleg619-server.virtnet: executing load_modules_local [ 1655.595348] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 1655.607614] alg: No test for adler32 (adler32-zlib) [ 1656.673786] Lustre: Lustre: Build Version: 2.17.57_98_gaf12cc4 [ 1656.900933] LNet: Added LNI 192.168.206.119@tcp [8/256/0/180] [ 1658.600310] Key type lgssc registered [ 1659.755164] Lustre: Echo OBD driver; http://www.lustre.org/ [ 1667.267530] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 1667.293201] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1668.748838] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 1673.696154] Lustre: DEBUG MARKER: oleg619-server.virtnet: executing set_default_debug -1 all [ 1680.873540] Lustre: DEBUG MARKER: oleg619-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 1688.472248] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 1689.138407] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 1693.256585] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x240000400:140 to 0x240000400:161) [ 1693.416195] Lustre: 63679:0:(osd_internal.h:1460:osd_trans_exec_op()) lustre-OST0000: opcode 8: before 509 < left 544, rollback = 9 [ 1693.425537] Lustre: 63679:0:(osd_handler.c:2090:osd_trans_dump_creds()) create: 32/128/6, destroy: 0/0/0 [ 1693.431130] Lustre: 63679:0:(osd_handler.c:2097:osd_trans_dump_creds()) attr_set: 0/0/0, xattr_set: 0/0/0 [ 1693.439966] Lustre: 63679:0:(osd_handler.c:2107:osd_trans_dump_creds()) write: 2/18/1, punch: 0/0/0, quota 1/3/0 [ 1693.454075] Lustre: 63679:0:(osd_handler.c:2114:osd_trans_dump_creds()) insert: 32/544/0, delete: 0/0/0 [ 1693.469273] Lustre: 63679:0:(osd_handler.c:2121:osd_trans_dump_creds()) ref_add: 0/0/0, ref_del: 0/0/0 [ 1697.589337] Lustre: DEBUG MARKER: oleg619-server.virtnet: executing set_default_debug -1 all [ 1707.643951] Lustre: DEBUG MARKER: oleg619-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 1712.789381] Lustre: DEBUG MARKER: check files after shrinking the MDT and OST filesystems [ 1754.394052] Lustre: server umount lustre-OST0000 complete [ 1769.266422] Lustre: server umount lustre-MDT0000 complete [ 1781.168310] Lustre: DEBUG MARKER: oleg619-server.virtnet: executing unload_modules_local [ 1783.826813] Key type lgssc unregistered [ 1784.132250] LNet: 65266:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 1784.146599] LNetError: 65266:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 1784.169055] LNet: Removed LNI 192.168.206.119@tcp [ 1784.963339] Key type .llcrypt unregistered [ 1784.968093] Key type ._llcrypt unregistered [ 1802.399067] Key type ._llcrypt registered [ 1802.401256] Key type .llcrypt registered [ 1802.546349] Lustre: DEBUG MARKER: oleg619-server.virtnet: executing set_hostid [ 1816.455466] Lustre: DEBUG MARKER: oleg619-server.virtnet: executing load_modules_local [ 1817.261658] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 1817.293408] alg: No test for adler32 (adler32-zlib) [ 1818.275460] Lustre: Lustre: Build Version: 2.17.57_98_gaf12cc4 [ 1818.481359] LNet: Added LNI 192.168.206.119@tcp [8/256/0/180] [ 1820.168801] Key type lgssc registered [ 1821.037693] Lustre: Echo OBD driver; http://www.lustre.org/ [ 1851.466766] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 1851.502381] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1852.763431] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 1852.839714] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 1852.894244] Lustre: lustre-MDT0000: new disk, initializing [ 1852.955697] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 1852.990067] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 1856.718945] Lustre: DEBUG MARKER: oleg619-server.virtnet: executing set_default_debug -1 all [ 1866.192115] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1866.325707] Lustre: 68419:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 1866.411923] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [ 1866.416588] Lustre: Skipped 1 previous similar message [ 1866.508292] Lustre: lustre-MDT0001: new disk, initializing [ 1866.600161] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 1866.642847] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [ 1866.654088] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 1870.394313] Lustre: DEBUG MARKER: oleg619-server.virtnet: executing set_default_debug -1 all [ 1878.793241] Lustre: DEBUG MARKER: oleg619-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 1884.215596] Lustre: DEBUG MARKER: oleg619-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 1890.448602] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 1890.817301] Lustre: lustre-OST0000: new disk, initializing [ 1890.824794] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 1890.835618] Lustre: 70389:0:(osd_compat.c:1353:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 1890.920344] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 1897.398368] Lustre: DEBUG MARKER: oleg619-server.virtnet: executing set_default_debug -1 all [ 1897.531220] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [ 1897.541261] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 1897.702344] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 1906.509917] Lustre: DEBUG MARKER: oleg619-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 1911.354265] Lustre: DEBUG MARKER: oleg619-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 [ 1911.552497] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [ 1916.091414] Lustre: DEBUG MARKER: oleg619-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid 50 [ 1916.371334] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [ 1918.441021] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 1918.443782] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 1918.448403] LustreError: Skipped 1 previous similar message [ 1918.449502] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 1918.489336] Lustre: Skipped 1 previous similar message [ 1923.555420] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 1923.560971] Lustre: Skipped 2 previous similar messages [ 1924.617740] Lustre: server umount lustre-OST0000 complete [ 1933.289351] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 1933.297713] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 1933.335946] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 1935.214113] Lustre: server umount lustre-MDT0000 complete [ 1939.833834] LustreError: 68413:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1787836387 with bad export cookie 4150295106386716734 [ 1939.842396] LustreError: MGC192.168.206.119@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 1939.843594] LustreError: 68413:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 1940.391385] Lustre: server umount lustre-MDT0001 complete [ 1951.742237] Lustre: DEBUG MARKER: == conf-sanity test 79: format MDT/OST without mgs option (should return errors) ========================================================== 09:13:15 (1787836395) [ 1964.222152] Lustre: DEBUG MARKER: oleg619-server.virtnet: executing load_modules_local [ 2008.025570] Lustre: DEBUG MARKER: oleg619-server.virtnet: executing set_hostid [ 2018.557529] Lustre: DEBUG MARKER: oleg619-server.virtnet: executing load_modules_local [ 2057.046533] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 2057.291975] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 2057.324293] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 2057.388583] Lustre: lustre-MDT0000: new disk, initializing [ 2057.445974] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 2057.473905] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 2062.329538] Lustre: DEBUG MARKER: oleg619-server.virtnet: executing set_default_debug -1 all [ 2073.030211] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 2073.226135] Lustre: 76410:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 2073.302552] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [ 2073.312231] Lustre: Skipped 1 previous similar message [ 2073.393272] Lustre: lustre-MDT0001: new disk, initializing [ 2073.496047] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 2073.560037] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [ 2073.574963] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 2078.374054] Lustre: DEBUG MARKER: oleg619-server.virtnet: executing set_default_debug -1 all [ 2088.007104] Lustre: DEBUG MARKER: oleg619-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 2093.046655] Lustre: DEBUG MARKER: oleg619-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 2100.392463] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 2100.631803] Lustre: lustre-OST0000: new disk, initializing [ 2100.641372] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 2100.649874] Lustre: 78378:0:(osd_compat.c:1353:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 2100.761411] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 2106.541429] Lustre: DEBUG MARKER: oleg619-server.virtnet: executing set_default_debug -1 all [ 2110.570049] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [ 2110.587357] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 2110.652933] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 2115.567428] Lustre: DEBUG MARKER: oleg619-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 2119.968547] Lustre: DEBUG MARKER: oleg619-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 [ 2120.118177] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [ 2124.415365] Lustre: DEBUG MARKER: oleg619-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid 50 [ 2124.589918] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [ 2126.310904] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2126.312961] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 2126.314910] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 2126.314916] Lustre: Skipped 1 previous similar message [ 2126.324036] Lustre: Skipped 2 previous similar messages [ 2126.327355] LustreError: Skipped 1 previous similar message [ 2131.429033] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 2131.435536] Lustre: Skipped 2 previous similar messages [ 2132.458165] Lustre: server umount lustre-OST0000 complete [ 2136.546274] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2136.570552] Lustre: Skipped 1 previous similar message [ 2141.665967] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 2141.671885] Lustre: Skipped 3 previous similar messages [ 2150.368232] Lustre: lustre-MDT0000 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 2150.633978] Lustre: server umount lustre-MDT0000 complete [ 2151.905733] LustreError: 76424:0:(ldlm_lib.c:1228:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 2151.917586] LustreError: 76424:0:(ldlm_lib.c:1228:target_handle_connect()) Skipped 4 previous similar messages [ 2153.847210] LustreError: 76404:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1787836601 with bad export cookie 4150295106386717616 [ 2153.863606] LustreError: MGC192.168.206.119@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 2153.864614] LustreError: 76404:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 2154.184356] Lustre: server umount lustre-MDT0001 complete [ 2162.956450] Lustre: DEBUG MARKER: == conf-sanity test 80: mgc import reconnect race ======== 09:16:48 (1787836608) [ 2169.920717] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 2170.347604] LustreError: 80793:0:(ldlm_lib.c:1228:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 2170.485832] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 2175.178655] Lustre: DEBUG MARKER: oleg619-server.virtnet: executing set_default_debug -1 all [ 2175.971979] LustreError: 80794:0:(ldlm_lib.c:1228:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 2181.102989] LustreError: 80793:0:(ldlm_lib.c:1228:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 2183.203792] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 2187.838786] Lustre: DEBUG MARKER: oleg619-server.virtnet: executing set_default_debug -1 all [ 2194.687076] Lustre: DEBUG MARKER: oleg619-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 2200.213609] Lustre: DEBUG MARKER: oleg619-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 2206.961806] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 2207.249993] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 2207.253879] Lustre: Skipped 1 previous similar message [ 2212.976614] Lustre: DEBUG MARKER: oleg619-server.virtnet: executing set_default_debug -1 all [ 2220.150935] Lustre: DEBUG MARKER: oleg619-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 2224.197627] Lustre: 83467:0:(genops.c:1774:obd_export_evict_by_uuid()) MGS: evicting c9aaa141-185f-48c5-bf18-8a94038e21f3 at adminstrative request [ 2225.637367] LustreError: MGC192.168.206.119@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 2225.665876] Lustre: *** cfs_fail_loc=906, val=2147483648*** [ 2235.885282] Lustre: MGS: Client c9aaa141-185f-48c5-bf18-8a94038e21f3 (at 0@lo) reconnecting [ 2235.902749] Lustre: Evicted from MGS (at 0@lo) after server handle changed from 0x3998cf68f938ef22 to 0x3998cf68f938f1c2 [ 2235.933394] Lustre: MGC192.168.206.119@tcp: Connection restored to 0@lo (at 0@lo) [ 2261.097961] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 2261.270112] Lustre: lustre-OST0001: new disk, initializing [ 2261.278224] Lustre: srv-lustre-OST0001: No data found on store. Initialize space. [ 2261.283834] Lustre: 83864:0:(osd_compat.c:1353:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 2267.376187] Lustre: DEBUG MARKER: oleg619-server.virtnet: executing set_default_debug -1 all [ 2268.252729] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x00000002c0000400-0x0000000300000400]:1:ost [ 2268.268159] Lustre: cli-lustre-OST0001-super: Allocated super-sequence [0x00000002c0000400-0x0000000300000400]:1:ost] [ 2268.366124] Lustre: lustre-OST0001-osc-MDT0000: update sequence from 0x100010000 to 0x2c0000401 [ 2276.634439] Lustre: DEBUG MARKER: oleg619-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0001-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 2284.000985] LustreError: lustre-OST0001-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 2284.012948] Lustre: lustre-OST0001-osc-MDT0001: Connection to lustre-OST0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2284.014564] LustreError: Skipped 1 previous similar message [ 2284.033772] Lustre: lustre-OST0001: Not available for connect from 0@lo (stopping) [ 2284.033779] Lustre: Skipped 2 previous similar messages [ 2285.578753] Lustre: server umount lustre-OST0001 complete [ 2289.122323] LustreError: 82702:0:(ldlm_lib.c:1228:target_handle_connect()) lustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 2289.141736] LustreError: 82702:0:(ldlm_lib.c:1228:target_handle_connect()) Skipped 1 previous similar message [ 2295.242371] Lustre: server umount lustre-OST0000 complete [ 2299.362542] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2299.377018] Lustre: Skipped 3 previous similar messages [ 2313.184280] Lustre: lustre-MDT0000 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 2313.385513] Lustre: server umount lustre-MDT0000 complete [ 2314.724070] LustreError: 80793:0:(ldlm_lib.c:1228:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 2314.743126] LustreError: 80793:0:(ldlm_lib.c:1228:target_handle_connect()) Skipped 3 previous similar messages [ 2317.101303] LustreError: 80775:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1787836764 with bad export cookie 4150295106386719170 [ 2317.105287] LustreError: MGC192.168.206.119@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 2317.111118] LustreError: 80775:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 2317.721681] Lustre: server umount lustre-MDT0001 complete [ 2327.059846] Lustre: DEBUG MARKER: == conf-sanity test 81: sparse OST indexing ============== 09:19:32 (1787836772) [ 2328.658049] Lustre: DEBUG MARKER: SKIP: conf-sanity test_81 needs >= 3 OSTs [ 2330.301510] Lustre: DEBUG MARKER: == conf-sanity test 82a: specify OSTs for file (succeed) or directory (succeed) ========================================================== 09:19:35 (1787836775) [ 2331.933825] Lustre: DEBUG MARKER: SKIP: conf-sanity test_82a needs >= 3 OSTs [ 2333.744675] Lustre: DEBUG MARKER: == conf-sanity test 82b: specify OSTs for file with --pool and --ost-list options ========================================================== 09:19:38 (1787836778) [ 2335.736294] Lustre: DEBUG MARKER: SKIP: conf-sanity test_82b needs >= 4 OSTs [ 2337.695575] Lustre: DEBUG MARKER: == conf-sanity test 82c: specify sparse OSTs for setstripe (should not crash) ========================================================== 09:19:42 (1787836782) [ 2339.693886] Lustre: DEBUG MARKER: SKIP: conf-sanity test_82c needs >= 4 OSTs [ 2341.860615] Lustre: DEBUG MARKER: == conf-sanity test 83: ENOSPACE on OST doesn't cause message VFS: Busy inodes after unmount ... ========================================================== 09:19:46 (1787836786) [ 2343.814709] Lustre: DEBUG MARKER: mount the OST /dev/mapper/ost1_flakey as a ldiskfs filesystem [ 2350.567094] loop: module loaded [ 2350.605421] loop0: detected capacity change from 0 to 4294967296 [ 2350.665252] LDISKFS-fs (loop0): mounted filesystem with ordered data mode. Opts: (null) [ 2352.389272] Lustre: DEBUG MARKER: run llverfs in partial mode on the OST ldiskfs /mnt/lustre-ost1 [ 2355.899664] Lustre: DEBUG MARKER: oleg619-server.virtnet: executing run_llverfs /mnt/lustre-ost1 -vpl no [ 2360.531895] Lustre: DEBUG MARKER: unmount the OST /dev/mapper/ost1_flakey [ 2390.953504] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 2391.004955] LustreError: 87993:0:(obd_config.c:845:class_setup()) setup lustre-OST0000-osd failed (-28) [ 2391.011384] LustreError: 87993:0:(obd_mount.c:259:lustre_start_simple()) lustre-OST0000-osd setup error -28 [ 2391.016652] LustreError: 87993:0:(tgt_mount.c:2579:server_fill_super()) Unable to start osd on /dev/mapper/ost1_flakey: -28 [ 2391.023442] LustreError: 87993:0:(super25.c:179:lustre_fill_super()) llite: Unable to mount : rc = -28 [ 2411.801795] Lustre: DEBUG MARKER: oleg619-server.virtnet: executing set_hostid [ 2421.537279] Lustre: DEBUG MARKER: oleg619-server.virtnet: executing load_modules_local [ 2463.754708] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 2464.227548] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 2464.321611] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 2464.461784] Lustre: lustre-MDT0000: new disk, initializing [ 2464.611113] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 2464.624679] Lustre: Skipped 1 previous similar message [ 2464.640475] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 2469.991515] Lustre: DEBUG MARKER: oleg619-server.virtnet: executing set_default_debug -1 all [ 2482.061618] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 2482.226964] Lustre: 90861:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 2482.618272] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 2487.313487] Lustre: DEBUG MARKER: oleg619-server.virtnet: executing set_default_debug -1 all [ 2497.499729] Lustre: DEBUG MARKER: oleg619-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 2503.492623] Lustre: DEBUG MARKER: oleg619-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 2510.134543] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 2510.480049] Lustre: 92829:0:(osd_compat.c:1353:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 2516.622360] Lustre: DEBUG MARKER: oleg619-server.virtnet: executing set_default_debug -1 all [ 2519.119464] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 2519.281318] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 2526.389503] Lustre: DEBUG MARKER: oleg619-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 2531.101620] Lustre: DEBUG MARKER: oleg619-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 [ 2531.274237] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [ 2536.049696] Lustre: DEBUG MARKER: oleg619-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid 50 [ 2536.278895] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [ 2540.005326] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2540.007061] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 2540.028502] Lustre: Skipped 1 previous similar message [ 2540.036369] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 2540.036920] Lustre: Skipped 11 previous similar messages [ 2545.033386] Lustre: server umount lustre-OST0000 complete [ 2549.219237] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 2563.040713] Lustre: lustre-MDT0000 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 2563.365883] Lustre: server umount lustre-MDT0000 complete [ 2564.577411] LustreError: 92837:0:(ldlm_lib.c:1228:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 2564.596389] LustreError: 92837:0:(ldlm_lib.c:1228:target_handle_connect()) Skipped 1 previous similar message [ 2566.892186] LustreError: 90851:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1787837014 with bad export cookie 4150295106386719786 [ 2566.896538] LustreError: MGC192.168.206.119@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 2566.913319] LustreError: 90851:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 2567.145173] Lustre: server umount lustre-MDT0001 complete [ 2576.871147] Lustre: DEBUG MARKER: == conf-sanity test 84: check recovery_hard_time ========= 09:23:42 (1787837022) [ 2584.149675] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 2589.399059] Lustre: DEBUG MARKER: oleg619-server.virtnet: executing set_default_debug -1 all [ 2599.404409] LustreError: 95244:0:(ldlm_lib.c:1228:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 2599.442717] LustreError: 95244:0:(ldlm_lib.c:1228:target_handle_connect()) Skipped 3 previous similar messages [ 2599.959043] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 2600.421116] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-60 [ 2600.429095] Lustre: Skipped 3 previous similar messages [ 2605.879209] Lustre: DEBUG MARKER: oleg619-server.virtnet: executing set_default_debug -1 all [ 2615.289143] Lustre: DEBUG MARKER: oleg619-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 2623.124807] Lustre: DEBUG MARKER: oleg619-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 2630.283724] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 2636.585689] Lustre: DEBUG MARKER: oleg619-server.virtnet: executing set_default_debug -1 all [ 2644.493788] Lustre: DEBUG MARKER: oleg619-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 2649.132713] Lustre: DEBUG MARKER: oleg619-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 [ 2654.795264] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 5 sec [ 2660.557385] Lustre: DEBUG MARKER: oleg619-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid 50 [ 2660.800902] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [ 2667.968764] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 2668.131656] Lustre: lustre-OST0001: new disk, initializing [ 2668.133761] Lustre: Skipped 2 previous similar messages [ 2668.136711] Lustre: srv-lustre-OST0001: No data found on store. Initialize space. [ 2668.142962] Lustre: Skipped 3 previous similar messages [ 2668.150509] Lustre: 98653:0:(osd_compat.c:1353:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 2673.741443] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x00000002c0000400-0x0000000300000400]:1:ost [ 2673.753314] Lustre: Skipped 2 previous similar messages [ 2673.762581] Lustre: cli-lustre-OST0001-super: Allocated super-sequence [0x00000002c0000400-0x0000000300000400]:1:ost] [ 2673.899663] Lustre: lustre-OST0001-osc-MDT0000: update sequence from 0x100010000 to 0x2c0000401 [ 2674.646324] Lustre: DEBUG MARKER: oleg619-server.virtnet: executing set_default_debug -1 all [ 2683.457517] Lustre: DEBUG MARKER: oleg619-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0001-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 2687.825969] Lustre: DEBUG MARKER: oleg619-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0001-osc-MDT0000.ost_server_uuid 50 [ 2688.052403] Lustre: DEBUG MARKER: os[cp].lustre-OST0001-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [ 2692.611692] Lustre: DEBUG MARKER: oleg619-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0001-osc-MDT0001.ost_server_uuid 50 [ 2692.904289] Lustre: DEBUG MARKER: os[cp].lustre-OST0001-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [ 2714.460492] Lustre: DEBUG MARKER: mds1 REPLAY BARRIER on lustre-MDT0000 [ 2739.211092] Lustre: Failing over lustre-MDT0000 [ 2739.736943] Lustre: server umount lustre-MDT0000 complete [ 2740.718219] Lustre: lustre-MDT0000-lwp-OST0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2740.722972] LustreError: 95240:0:(ldlm_lib.c:1228:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 2740.779770] Lustre: Skipped 5 previous similar messages [ 2740.804087] LustreError: 95240:0:(ldlm_lib.c:1228:target_handle_connect()) Skipped 3 previous similar messages [ 2757.092369] Lustre: 66644:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1787837188/real 1787837188] req@000000009b3e469c x1874682200728192/t0(0) o400->MGC192.168.206.119@tcp@0@lo:26/25 lens 224/224 e 0 to 1 dl 1787837204 ref 1 fl Rpc:XNQr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 [ 2757.108403] LustreError: MGC192.168.206.119@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 2763.147522] LDISKFS-fs (dm-0): 4 truncates cleaned up [ 2763.149698] LDISKFS-fs (dm-0): recovery complete [ 2763.157053] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 2766.306044] LustreError: 66643:0:(client.c:1404:ptlrpc_import_delay_req()) @@@ invalidate in flight req@0000000045133e7d x1874682200736512/t0(0) o250->MGC192.168.206.119@tcp@0@lo:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl Rpc:NQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 [ 2766.721329] Lustre: lustre-MDT0000: in recovery but waiting for the first client to connect [ 2767.836953] Lustre: lustre-MDT0000: Will be in recovery for at least 1:00, or until 3 clients reconnect [ 2767.838343] Lustre: 95240:0:(ldlm_lib.c:2148:extend_recovery_timer()) lustre-MDT0000: extended recovery timer reached hard limit: 60, extend: 0 [ 2771.192757] Lustre: DEBUG MARKER: oleg619-server.virtnet: executing set_default_debug -1 all [ 2771.939077] Lustre: 95975:0:(ldlm_lib.c:2148:extend_recovery_timer()) lustre-MDT0000: extended recovery timer reached hard limit: 60, extend: 0 [ 2771.951470] Lustre: 95975:0:(ldlm_lib.c:2148:extend_recovery_timer()) Skipped 1 previous similar message [ 2771.969851] Lustre: lustre-MDT0000-lwp-MDT0001: Connection restored to 0@lo (at 0@lo) [ 2772.049249] LustreError: 101507:0:(ldlm_lib.c:2761:replay_request_or_update()) cfs_fail_timeout id 709 sleeping for 300ms [ 2772.385457] LustreError: 101507:0:(ldlm_lib.c:2761:replay_request_or_update()) cfs_fail_timeout id 709 awake [ 2772.736873] LustreError: 101507:0:(ldlm_lib.c:2761:replay_request_or_update()) cfs_fail_timeout id 709 sleeping for 300ms [ 2772.757282] LustreError: 101507:0:(ldlm_lib.c:2761:replay_request_or_update()) Skipped 1 previous similar message [ 2773.081417] LustreError: 101507:0:(ldlm_lib.c:2761:replay_request_or_update()) cfs_fail_timeout id 709 awake [ 2773.100917] LustreError: 101507:0:(ldlm_lib.c:2761:replay_request_or_update()) Skipped 1 previous similar message [ 2773.112817] Lustre: 101507:0:(ldlm_lib.c:2148:extend_recovery_timer()) lustre-MDT0000: extended recovery timer reached hard limit: 60, extend: 1 [ 2773.128253] Lustre: 101507:0:(ldlm_lib.c:2148:extend_recovery_timer()) Skipped 7 previous similar messages [ 2773.769692] LustreError: 101507:0:(ldlm_lib.c:2761:replay_request_or_update()) cfs_fail_timeout id 709 sleeping for 300ms [ 2773.777050] LustreError: 101507:0:(ldlm_lib.c:2761:replay_request_or_update()) Skipped 2 previous similar messages [ 2774.107815] LustreError: 101507:0:(ldlm_lib.c:2761:replay_request_or_update()) cfs_fail_timeout id 709 awake [ 2774.121212] LustreError: 101507:0:(ldlm_lib.c:2761:replay_request_or_update()) Skipped 2 previous similar messages [ 2775.394727] Lustre: 101507:0:(ldlm_lib.c:2148:extend_recovery_timer()) lustre-MDT0000: extended recovery timer reached hard limit: 60, extend: 1 [ 2775.407141] Lustre: 101507:0:(ldlm_lib.c:2148:extend_recovery_timer()) Skipped 6 previous similar messages [ 2776.050655] LustreError: 101507:0:(ldlm_lib.c:2761:replay_request_or_update()) cfs_fail_timeout id 709 sleeping for 300ms [ 2776.062493] LustreError: 101507:0:(ldlm_lib.c:2761:replay_request_or_update()) Skipped 6 previous similar messages [ 2776.392440] LustreError: 101507:0:(ldlm_lib.c:2761:replay_request_or_update()) cfs_fail_timeout id 709 awake [ 2776.404679] LustreError: 101507:0:(ldlm_lib.c:2761:replay_request_or_update()) Skipped 6 previous similar messages [ 2779.557300] Lustre: 101507:0:(ldlm_lib.c:2148:extend_recovery_timer()) lustre-MDT0000: extended recovery timer reached hard limit: 60, extend: 1 [ 2779.578206] Lustre: 101507:0:(ldlm_lib.c:2148:extend_recovery_timer()) Skipped 12 previous similar messages [ 2780.234420] LustreError: 101507:0:(ldlm_lib.c:2761:replay_request_or_update()) cfs_fail_timeout id 709 sleeping for 300ms [ 2780.247859] LustreError: 101507:0:(ldlm_lib.c:2761:replay_request_or_update()) Skipped 12 previous similar messages [ 2780.576224] LustreError: 101507:0:(ldlm_lib.c:2761:replay_request_or_update()) cfs_fail_timeout id 709 awake [ 2780.584908] LustreError: 101507:0:(ldlm_lib.c:2761:replay_request_or_update()) Skipped 12 previous similar messages [ 2787.862869] Lustre: 101507:0:(ldlm_lib.c:2148:extend_recovery_timer()) lustre-MDT0000: extended recovery timer reached hard limit: 60, extend: 1 [ 2787.890546] Lustre: 101507:0:(ldlm_lib.c:2148:extend_recovery_timer()) Skipped 25 previous similar messages [ 2788.531314] LustreError: 101507:0:(ldlm_lib.c:2761:replay_request_or_update()) cfs_fail_timeout id 709 sleeping for 300ms [ 2788.545580] LustreError: 101507:0:(ldlm_lib.c:2761:replay_request_or_update()) Skipped 25 previous similar messages [ 2788.864701] LustreError: 101507:0:(ldlm_lib.c:2761:replay_request_or_update()) cfs_fail_timeout id 709 awake [ 2788.874319] LustreError: 101507:0:(ldlm_lib.c:2761:replay_request_or_update()) Skipped 25 previous similar messages [ 2804.122563] Lustre: 101507:0:(ldlm_lib.c:2148:extend_recovery_timer()) lustre-MDT0000: extended recovery timer reached hard limit: 60, extend: 1 [ 2804.142667] Lustre: 101507:0:(ldlm_lib.c:2148:extend_recovery_timer()) Skipped 50 previous similar messages [ 2804.789249] LustreError: 101507:0:(ldlm_lib.c:2761:replay_request_or_update()) cfs_fail_timeout id 709 sleeping for 300ms [ 2804.805510] LustreError: 101507:0:(ldlm_lib.c:2761:replay_request_or_update()) Skipped 50 previous similar messages [ 2805.144180] LustreError: 101507:0:(ldlm_lib.c:2761:replay_request_or_update()) cfs_fail_timeout id 709 awake [ 2805.150118] LustreError: 101507:0:(ldlm_lib.c:2761:replay_request_or_update()) Skipped 50 previous similar messages [ 2827.208212] Lustre: 101507:0:(ldlm_lib.c:2457:target_recovery_overseer()) lustre-MDT0000 recovery is aborted by hard timeout [ 2827.215585] Lustre: 101507:0:(ldlm_lib.c:2467:target_recovery_overseer()) recovery is aborted, evict exports in recovery [ 2827.221265] Lustre: 101507:0:(genops.c:1601:class_disconnect_stale_exports()) lustre-MDT0000: disconnect stale client 45e8abfc-4d62-4c97-b0d7-08b806367782@192.168.206.19@tcp [ 2827.234943] Lustre: lustre-MDT0000: disconnecting 1 stale clients [ 2827.239890] Lustre: 101507:0:(ldlm_lib.c:1977:abort_req_replay_queue()) @@@ aborted: req@00000000c15c778a x1874682185846016/t0(8589934770) o35->45e8abfc-4d62-4c97-b0d7-08b806367782@192.168.206.19@tcp:305/0 lens 392/0 e 0 to 0 dl 1787837285 ref 1 fl Complete:/604/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:0 [ 2827.262600] Lustre: lustre-MDT0000-osd: cancel update llog [0x200000400:0x1:0x0] [ 2827.278144] Lustre: lustre-MDT0000: Denying connection for new client 45e8abfc-4d62-4c97-b0d7-08b806367782 (at 192.168.206.19@tcp), waiting for 3 known clients (2 recovered, 0 in progress, and 1 evicted) to recover in 0:00 [ 2827.280192] Lustre: lustre-MDT0001-osp-MDT0000: cancel update llog [0x240000401:0x1:0x0] [ 2827.337956] Lustre: lustre-MDT0000-osp-MDT0001: Connection restored to 0@lo (at 0@lo) [ 2827.338976] Lustre: lustre-MDT0000: Recovery over after 1:00, of 3 clients 2 recovered and 1 was evicted. [ 2827.343250] Lustre: Skipped 2 previous similar messages [ 2827.387121] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:78 to 0x280000401:577) [ 2827.390432] Lustre: lustre-OST0001: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x2c0000401:45 to 0x2c0000401:545) [ 2838.502670] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2838.512246] Lustre: Skipped 1 previous similar message [ 2838.517417] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 2838.522497] Lustre: Skipped 9 previous similar messages [ 2852.320201] Lustre: lustre-OST0000 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 2852.388489] Lustre: server umount lustre-OST0000 complete [ 2858.467187] LustreError: lustre-OST0001-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 2861.733890] Lustre: server umount lustre-OST0001 complete [ 2871.242280] Lustre: server umount lustre-MDT0000 complete [ 2874.342883] LustreError: 95239:0:(ldlm_lib.c:1228:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 2874.355348] LustreError: 95239:0:(ldlm_lib.c:1228:target_handle_connect()) Skipped 39 previous similar messages [ 2875.141022] LustreError: 95224:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1787837322 with bad export cookie 4150295106386770998 [ 2875.149155] LustreError: MGC192.168.206.119@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 2875.150410] LustreError: 95224:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 2875.551565] Lustre: server umount lustre-MDT0001 complete [ 2884.580780] Lustre: DEBUG MARKER: == conf-sanity test 85: osd_ost init: fail ea_fid_set ==== 09:28:49 (1787837329) [ 2892.663158] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 2892.673174] Lustre: *** cfs_fail_loc=197, val=0*** [ 2908.258174] LustreError: Failed to get MGS log lustre-sptlrpc and no local copy. [ 2913.377427] LustreError: 103480:0:(mgc_request.c:233:do_config_log_add()) MGC192.168.206.119@tcp: failed processing log, type 4: rc = -110 [ 2918.538841] Lustre: *** cfs_fail_loc=197, val=0*** [ 2918.548890] Lustre: Skipped 65 previous similar messages [ 2939.104700] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 2939.109399] Lustre: Skipped 3 previous similar messages [ 2944.486406] Lustre: DEBUG MARKER: oleg619-server.virtnet: executing set_default_debug -1 all [ 2950.854784] Lustre: DEBUG MARKER: oleg619-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 2952.851394] Lustre: server umount lustre-OST0000 complete [ 2960.721236] Lustre: DEBUG MARKER: == conf-sanity test 86: Replacing mkfs.lustre -G option == 09:30:06 (1787837406) [ 2980.554577] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 2984.813520] Lustre: DEBUG MARKER: oleg619-server.virtnet: executing set_default_debug -1 all [ 2993.558639] Lustre: DEBUG MARKER: oleg619-client.virtnet: executing wait_import_state_mount (FULL|IDLE) mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 2997.207221] LustreError: 105135:0:(osp_object.c:618:osp_attr_get()) lustre-MDT0001-osp-MDT0000: osp_attr_get update error [0x200000009:0x1:0x0]: rc = -5 [ 2997.224347] LustreError: 105135:0:(lod_sub_object.c:917:lod_sub_prep_llog()) lustre-MDT0000-mdtlov: can't get id from catalogs: rc = -5 [ 2997.230851] LustreError: 105135:0:(lod_dev.c:511:lod_sub_recovery_thread()) lustre-MDT0001-osp-MDT0000: get update log duration 17, retries 0, failed: rc = -5 [ 2999.633491] Lustre: server umount lustre-MDT0000 complete [ 3031.887358] Lustre: DEBUG MARKER: oleg619-server.virtnet: executing set_hostid [ 3042.726267] Lustre: DEBUG MARKER: oleg619-server.virtnet: executing load_modules_local [ 3076.045943] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 3076.227351] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 3076.260189] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 3076.346812] Lustre: lustre-MDT0000: new disk, initializing [ 3076.481064] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 3080.740290] Lustre: DEBUG MARKER: oleg619-server.virtnet: executing set_default_debug -1 all [ 3090.455464] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 3090.593420] Lustre: 109307:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 3090.921839] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 3094.950106] Lustre: DEBUG MARKER: oleg619-server.virtnet: executing set_default_debug -1 all [ 3103.022385] Lustre: DEBUG MARKER: oleg619-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 3107.825601] Lustre: DEBUG MARKER: oleg619-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 3113.555424] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 3113.760583] Lustre: 111276:0:(osd_compat.c:1353:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 3119.925591] Lustre: DEBUG MARKER: oleg619-server.virtnet: executing set_default_debug -1 all [ 3120.822209] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 3128.526090] Lustre: DEBUG MARKER: oleg619-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 3133.019353] Lustre: DEBUG MARKER: oleg619-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 [ 3133.250623] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [ 3137.604250] Lustre: DEBUG MARKER: oleg619-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid 50 [ 3137.789340] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [ 3141.602316] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 3141.612164] LustreError: Skipped 2 previous similar messages [ 3141.618033] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 3141.633340] Lustre: Skipped 4 previous similar messages [ 3141.639626] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 3141.646802] Lustre: Skipped 10 previous similar messages [ 3145.874309] Lustre: server umount lustre-OST0000 complete [ 3155.307737] Lustre: server umount lustre-MDT0000 complete [ 3156.961922] LustreError: 109318:0:(ldlm_lib.c:1228:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 3156.977189] LustreError: 109318:0:(ldlm_lib.c:1228:target_handle_connect()) Skipped 6 previous similar messages [ 3158.146763] LustreError: 109299:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1787837605 with bad export cookie 4150295106386781596 [ 3158.159449] LustreError: MGC192.168.206.119@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 3158.160148] LustreError: 109299:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 2 previous similar messages [ 3158.460280] Lustre: server umount lustre-MDT0001 complete [ 3166.705066] Lustre: DEBUG MARKER: == conf-sanity test 87: check if MDT inode can hold EAs with N stripes properly ========================================================== 09:33:31 (1787837611) [ 3175.282796] Lustre: DEBUG MARKER: oleg619-server.virtnet: executing unload_modules_local [ 3177.931200] Key type lgssc unregistered [ 3178.264044] LNet: 113765:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 3178.283369] LNetError: 113765:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 3178.308184] LNet: Removed LNI 192.168.206.119@tcp [ 3179.361400] Key type .llcrypt unregistered [ 3179.363385] Key type ._llcrypt unregistered [ 3196.050590] Key type ._llcrypt registered [ 3196.052937] Key type .llcrypt registered [ 3196.136866] Lustre: DEBUG MARKER: oleg619-server.virtnet: executing set_hostid [ 3210.424184] Lustre: DEBUG MARKER: oleg619-server.virtnet: executing load_modules_local [ 3211.463612] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 3211.483773] alg: No test for adler32 (adler32-zlib) [ 3212.517782] Lustre: Lustre: Build Version: 2.17.57_98_gaf12cc4 [ 3212.757509] LNet: Added LNI 192.168.206.119@tcp [8/256/0/180] [ 3214.448159] Key type lgssc registered [ 3215.655339] Lustre: Echo OBD driver; http://www.lustre.org/ [ 3253.248528] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 3253.260393] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 3254.531088] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 3254.552404] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 3254.624401] Lustre: lustre-MDT0000: new disk, initializing [ 3254.700358] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 3254.715311] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 3258.814264] Lustre: DEBUG MARKER: oleg619-server.virtnet: executing set_default_debug -1 all [ 3268.327098] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 3268.635525] Lustre: lustre-OST0000: new disk, initializing [ 3268.643167] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 3268.649739] Lustre: Skipped 1 previous similar message [ 3268.653928] Lustre: 117938:0:(osd_compat.c:1353:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 3268.734484] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 3269.338841] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:0:ost [ 3269.344719] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:0:ost] [ 3269.365977] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x240000400 [ 3275.052327] Lustre: DEBUG MARKER: oleg619-server.virtnet: executing set_default_debug -1 all [ 3287.344503] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 3287.588825] Lustre: lustre-OST0001: new disk, initializing [ 3287.593447] Lustre: srv-lustre-OST0001: No data found on store. Initialize space. [ 3287.606842] Lustre: 118803:0:(osd_compat.c:1353:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 3287.735862] Lustre: lustre-OST0001: Imperative Recovery not enabled, recovery window 60-180 [ 3294.364726] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:1:ost [ 3294.377933] Lustre: cli-lustre-OST0001-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:1:ost] [ 3294.401969] Lustre: lustre-OST0001-osc-MDT0000: update sequence from 0x100010000 to 0x280000400 [ 3294.702472] Lustre: DEBUG MARKER: oleg619-server.virtnet: executing set_default_debug -1 all [ 3304.932378] Lustre: lustre-MDT0000-lwp-OST0000: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 3304.969652] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 3304.980704] Lustre: Skipped 1 previous similar message [ 3309.515811] Lustre: server umount lustre-MDT0000 complete [ 3317.092468] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: (null) [ 3322.118918] LustreError: 117124:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1787837769 with bad export cookie 14419426986095880581 [ 3322.122168] LustreError: MGC192.168.206.119@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 3322.126898] LustreError: 117124:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 1 previous similar message [ 3322.247696] Lustre: server umount lustre-OST0000 complete [ 3325.722650] Lustre: server umount lustre-OST0001 complete [ 3333.874239] Lustre: DEBUG MARKER: == conf-sanity test 88a: tunefs.lustre persists a --mountfsoptions-only change ========================================================== 09:36:18 (1787837778) [ 3355.581339] Lustre: DEBUG MARKER: == conf-sanity test 88b: check the default mount options can be overridden ========================================================== 09:36:40 (1787837800) [ 3365.390601] Lustre: DEBUG MARKER: oleg619-server.virtnet: executing load_modules_local [ 3386.794819] Lustre: DEBUG MARKER: == conf-sanity test 89: check tunefs --param and --erase-param{s} options ========================================================== 09:37:12 (1787837832) [ 3422.563791] Lustre: DEBUG MARKER: oleg619-server.virtnet: executing set_hostid [ 3432.885482] Lustre: DEBUG MARKER: oleg619-server.virtnet: executing load_modules_local [ 3465.200633] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 3465.387596] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 3465.420061] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 3465.511437] Lustre: lustre-MDT0000: new disk, initializing [ 3465.579830] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 3465.601388] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 3469.812822] Lustre: DEBUG MARKER: oleg619-server.virtnet: executing set_default_debug -1 all [ 3479.198126] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 3479.418212] Lustre: 126560:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 3479.515179] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [ 3479.519907] Lustre: Skipped 1 previous similar message [ 3479.638291] Lustre: lustre-MDT0001: new disk, initializing [ 3479.787671] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 3479.863555] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [ 3479.882891] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 3483.985915] Lustre: DEBUG MARKER: oleg619-server.virtnet: executing set_default_debug -1 all [ 3492.919339] Lustre: DEBUG MARKER: oleg619-client.virtnet: executing wait_import_state_mount FULL failover.node=192.0.2.254@tcp0,mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 3498.644063] Lustre: DEBUG MARKER: oleg619-client.virtnet: executing wait_import_state_mount FULL failover.node=192.0.2.254@tcp0,mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 3505.477791] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 3505.728117] Lustre: lustre-OST0000: new disk, initializing [ 3505.731781] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 3505.740908] Lustre: 128529:0:(osd_compat.c:1353:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 3505.857169] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 3512.749704] Lustre: DEBUG MARKER: oleg619-server.virtnet: executing set_default_debug -1 all [ 3515.474538] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [ 3515.489639] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 3515.613188] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 3522.051662] Lustre: DEBUG MARKER: oleg619-client.virtnet: executing wait_import_state_mount (FULL|IDLE) failover.node=192.0.2.254@tcp0,osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 3527.305109] Lustre: DEBUG MARKER: oleg619-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 [ 3527.623250] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [ 3532.382203] Lustre: DEBUG MARKER: oleg619-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid 50 [ 3532.676413] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [ 3536.357134] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 3536.357976] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 3536.358967] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 3536.364662] LustreError: Skipped 1 previous similar message [ 3536.397331] Lustre: Skipped 2 previous similar messages [ 3540.844697] Lustre: server umount lustre-OST0000 complete [ 3546.593738] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 3546.596178] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 3546.609899] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 3546.616613] Lustre: Skipped 1 previous similar message [ 3546.636609] Lustre: Skipped 2 previous similar messages [ 3550.903333] Lustre: server umount lustre-MDT0000 complete [ 3554.384856] LustreError: 126552:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1787838002 with bad export cookie 14419426986095881806 [ 3554.385835] LustreError: MGC192.168.206.119@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 3554.408047] LustreError: 126552:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 2 previous similar messages [ 3554.624995] Lustre: server umount lustre-MDT0001 complete [ 3563.480649] Lustre: DEBUG MARKER: == conf-sanity test 90a: check max_mod_rpcs_in_flight is enforced ========================================================== 09:40:08 (1787838008) [ 3571.071764] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 3571.703838] LustreError: 130940:0:(ldlm_lib.c:1228:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 3571.735781] LustreError: 130940:0:(ldlm_lib.c:1228:target_handle_connect()) Skipped 3 previous similar messages [ 3571.795884] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 3575.972695] Lustre: DEBUG MARKER: oleg619-server.virtnet: executing set_default_debug -1 all [ 3576.808801] LustreError: 130941:0:(ldlm_lib.c:1228:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 3581.929339] LustreError: 130940:0:(ldlm_lib.c:1228:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 3584.624413] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 3589.711457] Lustre: DEBUG MARKER: oleg619-server.virtnet: executing set_default_debug -1 all [ 3597.068448] Lustre: DEBUG MARKER: oleg619-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 3602.692971] Lustre: DEBUG MARKER: oleg619-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 3609.413113] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 3609.516893] LustreError: 132762:0:(ldlm_lib.c:1228:target_handle_connect()) lustre-OST0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 3609.823739] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 3609.833824] Lustre: Skipped 1 previous similar message [ 3618.067511] Lustre: DEBUG MARKER: oleg619-server.virtnet: executing set_default_debug -1 all [ 3627.266734] Lustre: DEBUG MARKER: oleg619-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 3631.583412] Lustre: *** cfs_fail_loc=159, val=0*** [ 3647.990432] Lustre: lustre-MDT0000: Client 6238d956-5c82-49a5-ac53-20716982848c (at 192.168.206.19@tcp) reconnecting [ 3649.386953] Lustre: *** cfs_fail_loc=159, val=0*** [ 3649.388286] Lustre: Skipped 5 previous similar messages [ 3665.935105] Lustre: lustre-MDT0000: Client 6238d956-5c82-49a5-ac53-20716982848c (at 192.168.206.19@tcp) reconnecting [ 3671.529164] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 3671.536455] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 3671.558222] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 3672.741618] Lustre: server umount lustre-OST0000 complete [ 3677.154586] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 3677.161928] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 3677.178257] Lustre: Skipped 1 previous similar message [ 3677.186279] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 3677.193890] Lustre: Skipped 2 previous similar messages [ 3682.789086] LustreError: 133601:0:(ldlm_lib.c:1228:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 3682.802285] LustreError: 133601:0:(ldlm_lib.c:1228:target_handle_connect()) Skipped 1 previous similar message [ 3682.857407] Lustre: server umount lustre-MDT0000 complete [ 3686.380610] LustreError: 130920:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1787838134 with bad export cookie 14419426986095882688 [ 3686.387436] LustreError: 130920:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 3 previous similar messages [ 3686.390143] LustreError: MGC192.168.206.119@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 3686.719779] Lustre: server umount lustre-MDT0001 complete [ 3696.999342] Lustre: DEBUG MARKER: oleg619-server.virtnet: executing unload_modules_local [ 3699.486430] Key type lgssc unregistered [ 3699.750297] LNet: 134721:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 3699.761691] LNetError: 134721:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 3699.778340] LNet: Removed LNI 192.168.206.119@tcp [ 3700.597398] Key type .llcrypt unregistered [ 3700.599538] Key type ._llcrypt unregistered [ 3723.760344] Key type ._llcrypt registered [ 3723.764081] Key type .llcrypt registered [ 3723.853398] Lustre: DEBUG MARKER: oleg619-server.virtnet: executing load_modules_local [ 3724.765261] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 3724.886305] alg: No test for adler32 (adler32-zlib) [ 3725.996098] Lustre: Lustre: Build Version: 2.17.57_98_gaf12cc4 [ 3726.308157] LNet: Added LNI 192.168.206.119@tcp [8/256/0/180] [ 3728.000621] Key type lgssc registered [ 3729.031709] Lustre: Echo OBD driver; http://www.lustre.org/ [ 3737.324869] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 3737.351347] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 3738.830887] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 3743.752111] Lustre: DEBUG MARKER: oleg619-server.virtnet: executing set_default_debug -1 all [ 3748.327279] LustreError: 136133:0:(ldlm_lib.c:1228:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 3748.346743] LustreError: 136133:0:(ldlm_lib.c:1228:target_handle_connect()) Skipped 2 previous similar messages [ 3753.450784] LustreError: 136134:0:(ldlm_lib.c:1228:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 3753.924101] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 3754.550384] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 3759.281792] Lustre: DEBUG MARKER: oleg619-server.virtnet: executing set_default_debug -1 all [ 3766.967341] Lustre: DEBUG MARKER: oleg619-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 3772.766897] Lustre: DEBUG MARKER: oleg619-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 3779.442600] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 3779.874706] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 3783.978188] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:42 to 0x280000401:65) [ 3785.767428] Lustre: DEBUG MARKER: oleg619-server.virtnet: executing set_default_debug -1 all [ 3795.073437] Lustre: DEBUG MARKER: oleg619-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 3805.406955] Lustre: *** cfs_fail_loc=159, val=0*** [ 3822.056934] Lustre: lustre-MDT0000: Client 34ca36e2-74fe-4e9f-a97b-5b48857e0402 (at 192.168.206.19@tcp) reconnecting [ 3824.644184] Lustre: *** cfs_fail_loc=159, val=0*** [ 3824.650282] Lustre: Skipped 1 previous similar message [ 3841.040937] Lustre: lustre-MDT0001: Client 34ca36e2-74fe-4e9f-a97b-5b48857e0402 (at 192.168.206.19@tcp) reconnecting [ 3842.771119] Lustre: *** cfs_fail_loc=159, val=0*** [ 3842.773075] Lustre: Skipped 2 previous similar messages [ 3858.436995] Lustre: lustre-MDT0001: Client 34ca36e2-74fe-4e9f-a97b-5b48857e0402 (at 192.168.206.19@tcp) reconnecting [ 3864.260782] Lustre: 139176:0:(mdt_handler.c:64:mdt_max_mod_rpcs_per_client_set()) max_mod_rpcs_per_client is deprecated, set mdt.*.max_mod_rpcs_in_flight parameter instead [ 3867.728890] Lustre: *** cfs_fail_loc=159, val=0*** [ 3867.730509] Lustre: Skipped 4 previous similar messages [ 3883.508355] Lustre: lustre-MDT0000: Client 4ffba8f6-e5d5-4ed2-978e-e624f883f40c (at 192.168.206.19@tcp) reconnecting [ 3885.490897] Lustre: *** cfs_fail_loc=159, val=0*** [ 3885.496848] Lustre: Skipped 13 previous similar messages [ 3901.947893] Lustre: lustre-MDT0000: Client 4ffba8f6-e5d5-4ed2-978e-e624f883f40c (at 192.168.206.19@tcp) reconnecting [ 3903.214282] Lustre: 139464:0:(mdt_handler.c:64:mdt_max_mod_rpcs_per_client_set()) max_mod_rpcs_per_client is deprecated, set mdt.*.max_mod_rpcs_in_flight parameter instead [ 3909.095528] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 3909.100975] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 3909.105645] LustreError: Skipped 1 previous similar message [ 3909.147313] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 3912.165361] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 3912.168671] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 3912.176559] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 3914.152865] Lustre: server umount lustre-OST0000 complete [ 3918.308393] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 3918.324253] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 3918.350111] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 3918.363976] Lustre: Skipped 1 previous similar message [ 3922.404286] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 3922.418206] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 3924.265842] Lustre: server umount lustre-MDT0000 complete [ 3927.520990] LustreError: 139236:0:(ldlm_lib.c:1228:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 3927.536046] LustreError: 139236:0:(ldlm_lib.c:1228:target_handle_connect()) Skipped 1 previous similar message [ 3927.957662] LustreError: 137976:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1787838375 with bad export cookie 122209741977556501 [ 3927.958960] LustreError: MGC192.168.206.119@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 3927.973258] LustreError: 137976:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 1 previous similar message [ 3928.433268] Lustre: server umount lustre-MDT0001 complete [ 3940.281681] Lustre: DEBUG MARKER: oleg619-server.virtnet: executing unload_modules_local [ 3942.583116] Key type lgssc unregistered [ 3942.791692] LNet: 140520:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 3942.797466] LNetError: 140520:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 3942.822541] LNet: Removed LNI 192.168.206.119@tcp [ 3943.468779] Key type .llcrypt unregistered [ 3943.470276] Key type ._llcrypt unregistered [ 3965.592523] Key type ._llcrypt registered [ 3965.597413] Key type .llcrypt registered [ 3965.754248] Lustre: DEBUG MARKER: oleg619-server.virtnet: executing load_modules_local [ 3967.285296] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 3967.337168] alg: No test for adler32 (adler32-zlib) [ 3968.430653] Lustre: Lustre: Build Version: 2.17.57_98_gaf12cc4 [ 3968.706052] LNet: Added LNI 192.168.206.119@tcp [8/256/0/180] [ 3970.344216] Key type lgssc registered [ 3971.237250] Lustre: Echo OBD driver; http://www.lustre.org/ [ 3979.437849] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 3979.471236] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 3980.998032] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 3984.623584] Lustre: DEBUG MARKER: oleg619-server.virtnet: executing set_default_debug -1 all [ 3990.505642] LustreError: 141937:0:(ldlm_lib.c:1228:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 3990.540486] LustreError: 141937:0:(ldlm_lib.c:1228:target_handle_connect()) Skipped 2 previous similar messages [ 3994.624093] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 3995.306493] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 4001.733044] Lustre: DEBUG MARKER: oleg619-server.virtnet: executing set_default_debug -1 all [ 4009.449868] Lustre: DEBUG MARKER: oleg619-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 4014.528257] Lustre: DEBUG MARKER: oleg619-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 4020.000650] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 4020.201408] LustreError: 143759:0:(ldlm_lib.c:1228:target_handle_connect()) lustre-OST0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 4020.320571] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 4025.276707] Lustre: DEBUG MARKER: oleg619-server.virtnet: executing set_default_debug -1 all [ 4025.332843] Lustre: lustre-OST0000: new connection from lustre-MDT0001-mdtlov (cleaning up unused objects from 0x280000400:72 to 0x280000400:97) [ 4026.359992] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:85 to 0x280000401:129) [ 4031.774679] Lustre: DEBUG MARKER: oleg619-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 4038.056458] Lustre: 144521:0:(mdt_handler.c:64:mdt_max_mod_rpcs_per_client_set()) max_mod_rpcs_per_client is deprecated, set mdt.*.max_mod_rpcs_in_flight parameter instead [ 4040.565837] Lustre: 144570:0:(mdt_handler.c:64:mdt_max_mod_rpcs_per_client_set()) max_mod_rpcs_per_client is deprecated, set mdt.*.max_mod_rpcs_in_flight parameter instead [ 4041.840237] Lustre: Modifying parameter lustre.mdc.lustre-MDT0000-mdc-*.max_rpcs_in_flight=8 in log params [ 4042.639435] Lustre: Setting parameter lustre.mdc.lustre-MDT0000-mdc-*.max_mod_rpcs_in_flight=8 in log params [ 4045.887972] Lustre: 144715:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre/mdc.lustre-MDT0000-mdc-*.max_rpcs_in_flight=8 (mode = 0) failed: rc = -17 [ 4046.716372] Lustre: Modifying parameter lustre.mdc.lustre-MDT0000-mdc-*.max_mod_rpcs_in_flight=7 in log params [ 4047.546455] Lustre: 144814:0:(mdt_handler.c:64:mdt_max_mod_rpcs_per_client_set()) max_mod_rpcs_per_client is deprecated, set mdt.*.max_mod_rpcs_in_flight parameter instead [ 4051.941915] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 4051.943199] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4051.951261] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 4051.960531] Lustre: Skipped 1 previous similar message [ 4051.979310] Lustre: Skipped 1 previous similar message [ 4057.064227] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 4062.188306] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 4062.204347] Lustre: Skipped 2 previous similar messages [ 4066.272155] Lustre: lustre-OST0000 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 4066.492161] Lustre: server umount lustre-OST0000 complete [ 4071.906081] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 4071.916438] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4071.934778] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 4076.545484] Lustre: server umount lustre-MDT0000 complete [ 4077.538995] LustreError: 142867:0:(ldlm_lib.c:1228:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 4077.571976] LustreError: 142867:0:(ldlm_lib.c:1228:target_handle_connect()) Skipped 1 previous similar message [ 4080.047226] LustreError: 141917:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1787838527 with bad export cookie 17876048748398121161 [ 4080.059793] LustreError: MGC192.168.206.119@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 4080.060076] LustreError: 141917:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 3 previous similar messages [ 4080.501443] Lustre: server umount lustre-MDT0001 complete [ 4091.768898] Lustre: DEBUG MARKER: oleg619-server.virtnet: executing unload_modules_local [ 4094.937291] Key type lgssc unregistered [ 4095.261079] LNet: 145878:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 4095.276523] LNetError: 145878:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 4095.298238] LNet: Removed LNI 192.168.206.119@tcp [ 4096.324263] Key type .llcrypt unregistered [ 4096.327166] Key type ._llcrypt unregistered [ 4120.849975] Key type ._llcrypt registered [ 4120.870251] Key type .llcrypt registered [ 4121.009990] Lustre: DEBUG MARKER: oleg619-server.virtnet: executing load_modules_local [ 4122.195833] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 4122.248535] alg: No test for adler32 (adler32-zlib) [ 4123.324337] Lustre: Lustre: Build Version: 2.17.57_98_gaf12cc4 [ 4123.634946] LNet: Added LNI 192.168.206.119@tcp [8/256/0/180] [ 4125.312225] Key type lgssc registered [ 4126.402790] Lustre: Echo OBD driver; http://www.lustre.org/ [ 4133.676655] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 4133.717928] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 4135.108548] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 4139.108754] Lustre: DEBUG MARKER: oleg619-server.virtnet: executing set_default_debug -1 all [ 4144.626761] LustreError: 147290:0:(ldlm_lib.c:1228:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 4144.660262] LustreError: 147290:0:(ldlm_lib.c:1228:target_handle_connect()) Skipped 2 previous similar messages [ 4147.154657] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 4147.631189] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 4153.478489] Lustre: DEBUG MARKER: oleg619-server.virtnet: executing set_default_debug -1 all [ 4161.036579] Lustre: DEBUG MARKER: oleg619-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 4168.639631] Lustre: DEBUG MARKER: oleg619-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 4175.378967] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 4175.823487] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 4179.960028] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:85 to 0x280000401:161) [ 4181.929688] Lustre: DEBUG MARKER: oleg619-server.virtnet: executing set_default_debug -1 all [ 4189.309914] Lustre: DEBUG MARKER: oleg619-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 4192.253714] Lustre: lustre-OST0000: new connection from lustre-MDT0001-mdtlov (cleaning up unused objects from 0x280000400:72 to 0x280000400:129) [ 4199.358403] Lustre: *** cfs_fail_loc=159, val=0*** [ 4215.801811] Lustre: lustre-MDT0000: Client 4afa7ce5-0c1a-4ffc-a53e-0c6fb59454c5 (at 192.168.206.19@tcp) reconnecting [ 4233.196967] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 4233.203189] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4233.218454] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 4233.234896] Lustre: Skipped 1 previous similar message [ 4238.309863] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 4238.324654] Lustre: Skipped 2 previous similar messages [ 4238.743856] Lustre: server umount lustre-OST0000 complete [ 4243.427714] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4243.442890] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 4243.449587] Lustre: Skipped 1 previous similar message [ 4248.550044] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 4248.564294] Lustre: Skipped 1 previous similar message [ 4249.518288] Lustre: server umount lustre-MDT0000 complete [ 4253.670032] LustreError: 148237:0:(ldlm_lib.c:1228:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 4253.692830] LustreError: 148237:0:(ldlm_lib.c:1228:target_handle_connect()) Skipped 1 previous similar message [ 4254.570381] LustreError: 147272:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1787838702 with bad export cookie 4345807697629557066 [ 4254.576073] LustreError: MGC192.168.206.119@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 4254.589154] LustreError: 147272:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 2 previous similar messages [ 4255.151750] Lustre: server umount lustre-MDT0001 complete [ 4267.875533] Lustre: DEBUG MARKER: oleg619-server.virtnet: executing unload_modules_local [ 4271.137251] Key type lgssc unregistered [ 4271.495570] LNet: 150959:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 4271.506122] LNetError: 150959:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 4271.532135] LNet: Removed LNI 192.168.206.119@tcp [ 4272.514601] Key type .llcrypt unregistered [ 4272.519954] Key type ._llcrypt unregistered [ 4300.049017] Key type ._llcrypt registered [ 4300.051840] Key type .llcrypt registered [ 4300.169356] Lustre: DEBUG MARKER: oleg619-server.virtnet: executing load_modules_local [ 4301.649488] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 4301.673018] alg: No test for adler32 (adler32-zlib) [ 4302.827845] Lustre: Lustre: Build Version: 2.17.57_98_gaf12cc4 [ 4303.124820] LNet: Added LNI 192.168.206.119@tcp [8/256/0/180] [ 4304.816195] Key type lgssc registered [ 4306.418478] Lustre: Echo OBD driver; http://www.lustre.org/ [ 4313.836978] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 4313.856441] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 4315.293023] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 4320.114311] Lustre: DEBUG MARKER: oleg619-server.virtnet: executing set_default_debug -1 all [ 4325.862906] LustreError: 152375:0:(ldlm_lib.c:1228:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 4325.880090] LustreError: 152375:0:(ldlm_lib.c:1228:target_handle_connect()) Skipped 2 previous similar messages [ 4329.406646] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 4329.828682] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 4334.618121] Lustre: DEBUG MARKER: oleg619-server.virtnet: executing set_default_debug -1 all [ 4342.864911] Lustre: DEBUG MARKER: oleg619-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 4349.555859] Lustre: DEBUG MARKER: oleg619-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 4355.763218] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 4356.181230] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 4360.324774] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:170 to 0x280000401:193) [ 4361.680872] Lustre: DEBUG MARKER: oleg619-server.virtnet: executing set_default_debug -1 all [ 4370.209809] Lustre: DEBUG MARKER: oleg619-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 4374.657538] Lustre: lustre-OST0000: new connection from lustre-MDT0001-mdtlov (cleaning up unused objects from 0x280000400:72 to 0x280000400:161) [ 4395.365922] Lustre: lustre-MDT0001: evicting 5a67ada8-e33d-4c3e-a72c-a4dc45b413c5 (at 192.168.206.19@tcp) by administrative request [ 4395.373824] Lustre: Skipped 1 previous similar message [ 4426.218445] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 4426.223166] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4426.238754] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 4426.241418] Lustre: Skipped 1 previous similar message [ 4426.264990] Lustre: Skipped 1 previous similar message [ 4430.010539] Lustre: server umount lustre-OST0000 complete [ 4436.454328] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4436.458807] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 4436.473883] Lustre: Skipped 1 previous similar message [ 4436.488979] Lustre: Skipped 1 previous similar message [ 4441.569426] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 4441.597890] Lustre: Skipped 1 previous similar message [ 4446.693065] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 4446.704948] Lustre: Skipped 1 previous similar message [ 4448.224107] Lustre: lustre-MDT0000 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 4448.431865] Lustre: server umount lustre-MDT0000 complete [ 4451.811143] LustreError: 152370:0:(ldlm_lib.c:1228:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 4451.846268] LustreError: 152370:0:(ldlm_lib.c:1228:target_handle_connect()) Skipped 1 previous similar message [ 4452.251336] LustreError: 152357:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1787838900 with bad export cookie 12887738133009538299 [ 4452.258541] LustreError: MGC192.168.206.119@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 4452.538502] Lustre: server umount lustre-MDT0001 complete [ 4463.376622] Lustre: DEBUG MARKER: oleg619-server.virtnet: executing unload_modules_local [ 4465.946921] Key type lgssc unregistered [ 4466.284827] LNet: 156449:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 4466.293840] LNetError: 156449:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 4466.308593] LNet: Removed LNI 192.168.206.119@tcp [ 4467.114527] Key type .llcrypt unregistered [ 4467.119422] Key type ._llcrypt unregistered [ 4546.619464] Key type ._llcrypt registered [ 4546.623194] Key type .llcrypt registered [ 4546.802675] Lustre: DEBUG MARKER: oleg619-server.virtnet: executing load_modules_local [ 4548.046052] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 4548.113658] alg: No test for adler32 (adler32-zlib) [ 4549.186167] Lustre: Lustre: Build Version: 2.17.57_98_gaf12cc4 [ 4549.562238] LNet: Added LNI 192.168.206.119@tcp [8/256/0/180] [ 4551.256185] Key type lgssc registered [ 4552.665955] Lustre: Echo OBD driver; http://www.lustre.org/ [ 4561.015453] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 4561.029245] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 4562.539393] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 4566.892823] Lustre: DEBUG MARKER: oleg619-server.virtnet: executing set_default_debug -1 all [ 4572.647744] LustreError: 158941:0:(ldlm_lib.c:1228:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 4572.668897] LustreError: 158941:0:(ldlm_lib.c:1228:target_handle_connect()) Skipped 2 previous similar messages [ 4575.920677] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 4576.357316] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 4580.858101] Lustre: DEBUG MARKER: oleg619-server.virtnet: executing set_default_debug -1 all [ 4588.442764] Lustre: DEBUG MARKER: oleg619-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 4594.155925] Lustre: DEBUG MARKER: oleg619-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 4601.235068] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 4601.755250] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 4602.803283] Lustre: lustre-OST0000: new connection from lustre-MDT0001-mdtlov (cleaning up unused objects from 0x280000400:72 to 0x280000400:193) [ 4607.310663] Lustre: DEBUG MARKER: oleg619-server.virtnet: executing set_default_debug -1 all [ 4607.919363] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:195 to 0x280000401:225) [ 4614.804800] Lustre: DEBUG MARKER: oleg619-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 4628.452699] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 4628.463405] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4628.465708] LustreError: Skipped 1 previous similar message [ 4628.468262] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 4628.497564] Lustre: Skipped 1 previous similar message [ 4630.722187] Lustre: server umount lustre-OST0000 complete [ 4637.667679] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 4637.680036] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4637.707126] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 4637.714914] Lustre: Skipped 1 previous similar message [ 4638.691686] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4638.707044] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 4638.709538] Lustre: Skipped 1 previous similar message [ 4640.099444] LustreError: 161731:0:(ldlm_resource.c:1207:ldlm_resource_complain()) MGS: namespace resource [0x70616d65646f6e:0x4:0x0].0x0 (0000000085a7400d) refcount nonzero (1) after lock cleanup; forcing cleanup. [ 4640.239577] Lustre: server umount lustre-MDT0000 complete [ 4643.696073] LustreError: 158922:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1787839091 with bad export cookie 8891147109968438100 [ 4643.702844] LustreError: MGC192.168.206.119@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 4643.710980] LustreError: 158922:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 4644.071415] Lustre: server umount lustre-MDT0001 complete [ 4653.779382] Lustre: DEBUG MARKER: oleg619-server.virtnet: executing unload_modules_local [ 4656.400633] Key type lgssc unregistered [ 4656.716457] LNet: 162486:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 4656.722715] LNetError: 162486:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 4656.740317] LNet: Removed LNI 192.168.206.119@tcp [ 4657.645966] Key type .llcrypt unregistered [ 4657.648574] Key type ._llcrypt unregistered [ 4681.410396] Key type ._llcrypt registered [ 4681.414490] Key type .llcrypt registered [ 4681.530373] Lustre: DEBUG MARKER: oleg619-server.virtnet: executing load_modules_local [ 4682.391860] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 4682.554740] alg: No test for adler32 (adler32-zlib) [ 4683.627819] Lustre: Lustre: Build Version: 2.17.57_98_gaf12cc4 [ 4683.843582] LNet: Added LNI 192.168.206.119@tcp [8/256/0/180] [ 4685.560200] Key type lgssc registered [ 4686.557717] Lustre: Echo OBD driver; http://www.lustre.org/ [ 4700.393992] Lustre: DEBUG MARKER: == conf-sanity test 100: check lshowmount lists MGS, MDT, OST and 0@lo ========================================================== 09:59:05 (1787839145) [ 4714.696284] Lustre: DEBUG MARKER: oleg619-server.virtnet: executing set_hostid [ 4722.701157] Lustre: DEBUG MARKER: oleg619-server.virtnet: executing load_modules_local [ 4751.284340] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 4751.306276] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 4752.540350] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 4752.589238] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 4752.677880] Lustre: lustre-MDT0000: new disk, initializing [ 4752.754124] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 4752.768271] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 4755.859743] Lustre: DEBUG MARKER: oleg619-server.virtnet: executing set_default_debug -1 all [ 4764.561433] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 4764.678889] Lustre: 166749:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 4764.740060] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [ 4764.743373] Lustre: Skipped 1 previous similar message [ 4764.809659] Lustre: lustre-MDT0001: new disk, initializing [ 4764.893072] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 4764.936471] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [ 4764.950027] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 4769.089788] Lustre: DEBUG MARKER: oleg619-server.virtnet: executing set_default_debug -1 all [ 4777.701585] Lustre: DEBUG MARKER: oleg619-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 4783.168423] Lustre: DEBUG MARKER: oleg619-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 4791.028962] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 4791.286420] Lustre: lustre-OST0000: new disk, initializing [ 4791.293268] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 4791.304127] Lustre: 168715:0:(osd_compat.c:1353:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 4791.403223] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 4798.370568] Lustre: DEBUG MARKER: oleg619-server.virtnet: executing set_default_debug -1 all [ 4800.112559] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [ 4800.132666] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 4800.215431] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 4807.108927] Lustre: DEBUG MARKER: oleg619-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 4815.843400] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 4815.852050] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4815.864057] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 4815.873761] Lustre: Skipped 1 previous similar message [ 4820.964290] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 4820.975161] Lustre: Skipped 1 previous similar message [ 4821.812541] Lustre: server umount lustre-OST0000 complete [ 4826.082735] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4826.094289] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 4826.099851] Lustre: Skipped 2 previous similar messages [ 4826.113914] Lustre: Skipped 1 previous similar message [ 4831.202287] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 4831.213409] Lustre: Skipped 1 previous similar message [ 4836.324042] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 4836.328974] Lustre: Skipped 1 previous similar message [ 4839.392789] Lustre: lustre-MDT0000 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 4839.753486] Lustre: server umount lustre-MDT0000 complete [ 4842.949020] LustreError: 166742:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1787839290 with bad export cookie 12297367164863485092 [ 4842.960147] LustreError: MGC192.168.206.119@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 4843.250654] Lustre: server umount lustre-MDT0001 complete [ 4851.350467] Lustre: DEBUG MARKER: oleg619-server.virtnet: executing unload_modules_local [ 4853.560399] Key type lgssc unregistered [ 4853.824404] LNet: 170637:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 4853.834918] LNetError: 170637:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 4853.847933] LNet: Removed LNI 192.168.206.119@tcp [ 4854.593321] Key type .llcrypt unregistered [ 4854.596825] Key type ._llcrypt unregistered [ 4878.097637] Key type ._llcrypt registered [ 4878.100397] Key type .llcrypt registered [ 4878.192168] Lustre: DEBUG MARKER: oleg619-server.virtnet: executing load_modules_local [ 4878.960150] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 4878.985846] alg: No test for adler32 (adler32-zlib) [ 4880.122818] Lustre: Lustre: Build Version: 2.17.57_98_gaf12cc4 [ 4880.408969] LNet: Added LNI 192.168.206.119@tcp [8/256/0/180] [ 4882.114836] Key type lgssc registered [ 4883.306880] Lustre: Echo OBD driver; http://www.lustre.org/ [ 4891.012118] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 4891.043453] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 4892.467698] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 4897.490875] Lustre: DEBUG MARKER: oleg619-server.virtnet: executing set_default_debug -1 all [ 4902.884174] LustreError: 172048:0:(ldlm_lib.c:1228:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 4902.907121] LustreError: 172048:0:(ldlm_lib.c:1228:target_handle_connect()) Skipped 2 previous similar messages [ 4906.023610] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 4906.331863] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 4910.117797] Lustre: DEBUG MARKER: oleg619-server.virtnet: executing set_default_debug -1 all [ 4918.284112] Lustre: DEBUG MARKER: oleg619-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 4924.741185] Lustre: DEBUG MARKER: oleg619-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 4931.415281] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 4931.779898] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 4937.908927] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:3 to 0x280000401:33) [ 4938.556103] Lustre: DEBUG MARKER: oleg619-server.virtnet: executing set_default_debug -1 all [ 4946.632695] Lustre: DEBUG MARKER: oleg619-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 4953.736516] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 4953.758976] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4953.774239] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 4953.777365] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4953.794011] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 4954.775178] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 4954.786031] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4954.800989] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 4954.806375] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4954.817698] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 4955.846848] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 4955.864676] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4955.880930] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 4955.887212] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4955.904477] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 4956.768644] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4957.733737] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4958.709669] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 4958.719861] Lustre: Skipped 2 previous similar messages [ 4958.732812] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4958.745314] Lustre: Skipped 2 previous similar messages [ 4958.754539] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 4958.760619] Lustre: Skipped 2 previous similar messages [ 4958.769923] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4958.780439] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 4958.790614] Lustre: Skipped 2 previous similar messages [ 4959.653088] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4960.526151] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4961.432049] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4962.169911] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4963.019694] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 4963.028743] Lustre: Skipped 4 previous similar messages [ 4963.049612] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4963.060449] Lustre: Skipped 4 previous similar messages [ 4963.068080] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 4963.076079] Lustre: Skipped 4 previous similar messages [ 4963.079938] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4963.093251] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 4963.098786] Lustre: Skipped 4 previous similar messages [ 4963.871093] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4964.680094] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4965.456563] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4966.407589] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4967.271965] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4968.218277] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4969.044246] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4969.961136] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4970.728235] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4971.577445] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 4971.581622] Lustre: Skipped 9 previous similar messages [ 4971.597827] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4971.605687] Lustre: Skipped 9 previous similar messages [ 4971.608663] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 4971.612757] Lustre: Skipped 9 previous similar messages [ 4971.616811] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4971.623963] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 4971.632157] Lustre: Skipped 9 previous similar messages [ 4972.448766] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4973.455558] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4974.303555] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4975.118679] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4975.904285] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4976.793470] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4977.561402] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4978.469408] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4979.304724] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4980.116524] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4980.877940] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4981.791984] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4982.540076] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4983.386652] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4984.213830] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4984.984108] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4985.794204] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4986.709973] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4987.489788] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4988.252309] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 4988.257859] Lustre: Skipped 19 previous similar messages [ 4988.274585] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4988.281536] Lustre: Skipped 19 previous similar messages [ 4988.284501] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 4988.286960] Lustre: Skipped 19 previous similar messages [ 4988.289665] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4988.294868] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 4988.298121] Lustre: Skipped 19 previous similar messages [ 4989.116326] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4989.996640] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4990.904075] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4991.728963] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4992.515876] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4993.371168] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4994.167733] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4995.020796] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4995.874649] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4996.694533] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4997.496476] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4998.395462] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4999.177833] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5000.066869] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5000.901385] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5001.704044] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5002.653996] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5003.621448] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5004.360745] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5005.126137] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5005.873247] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5006.603683] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5007.427669] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5008.235401] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5009.046660] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5010.061729] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5010.777986] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5011.606087] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5012.313831] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5013.048458] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5013.815308] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5014.568386] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5015.442241] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5016.162766] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5017.027125] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5017.741561] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5018.732350] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5019.460684] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5020.316023] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 5020.323494] Lustre: Skipped 38 previous similar messages [ 5020.340758] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5020.348584] Lustre: Skipped 38 previous similar messages [ 5020.353741] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 5020.357840] Lustre: Skipped 38 previous similar messages [ 5020.361965] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5020.374797] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 5020.379501] Lustre: Skipped 38 previous similar messages [ 5021.212675] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5021.994315] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5022.878512] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5023.903637] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5025.059162] LustreError: 172068:0:(osp_precreate.c:655:osp_precreate_send()) lustre-OST0000-osc-MDT0000: can't precreate: rc = -5 [ 5025.082126] LustreError: 172068:0:(osp_precreate.c:1359:osp_precreate_thread()) lustre-OST0000-osc-MDT0000: cannot precreate objects: rc = -5 [ 5025.097643] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5026.414580] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5027.679597] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5029.090663] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5030.311864] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5031.404803] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5032.352294] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5033.520471] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5034.506094] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5035.566214] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5036.863500] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5038.229987] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5038.982243] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5040.001285] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5040.916723] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5041.786731] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5043.002490] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5043.927747] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5044.913066] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5045.765286] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5046.614632] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5047.407499] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5048.273857] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5049.087400] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5049.904388] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5050.684147] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5051.624666] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5052.445353] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5053.309894] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5054.177789] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5055.049456] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5055.763979] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5056.608437] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5057.341679] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5058.124932] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5058.955093] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5059.930806] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5060.712056] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5061.688850] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5062.539747] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5063.355679] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5064.093241] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5064.870198] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5065.741917] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5066.579628] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5067.294222] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5068.327873] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5069.325270] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5070.216381] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5071.258745] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5072.128708] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5072.955967] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5073.668335] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5074.405454] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5075.278190] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5076.227041] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5077.036267] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5077.839899] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5078.549426] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5079.318421] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5080.017813] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5080.784084] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5081.500301] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5082.444919] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5083.134920] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5083.788417] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5084.633364] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 5084.639448] Lustre: Skipped 70 previous similar messages [ 5084.665513] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5084.683352] Lustre: Skipped 70 previous similar messages [ 5084.690776] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 5084.698098] Lustre: Skipped 70 previous similar messages [ 5084.703539] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5084.715315] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 5084.736394] Lustre: Skipped 70 previous similar messages [ 5085.626206] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5086.577722] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5087.704470] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5088.625560] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5089.542258] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5090.314479] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5091.428558] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5092.706528] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5093.943160] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5095.125350] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5095.981712] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5096.854862] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5097.807441] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5098.774972] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5099.959807] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5100.821270] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5101.812190] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5102.777718] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5103.654560] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5104.424403] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5105.311302] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5106.040934] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5107.109738] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5107.969931] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5108.874221] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5109.674679] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5110.538278] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5111.318740] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5112.284727] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5113.320234] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5114.182064] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5115.067827] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5116.038923] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5116.702489] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5117.618101] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5118.722950] LustreError: 172068:0:(osp_precreate.c:655:osp_precreate_send()) lustre-OST0000-osc-MDT0000: can't precreate: rc = -5 [ 5118.735801] LustreError: 172068:0:(osp_precreate.c:1359:osp_precreate_thread()) lustre-OST0000-osc-MDT0000: cannot precreate objects: rc = -5 [ 5118.746130] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5119.569024] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5120.692721] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5121.941435] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5123.095984] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5123.985224] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5124.769427] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5125.764827] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5126.627478] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5127.622127] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5128.844581] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5129.954500] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5130.919202] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5131.996925] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5133.004985] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5134.073451] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5135.249054] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5136.267719] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5137.467697] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5138.575588] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5139.730794] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5140.917958] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5141.753735] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5142.552191] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5143.449387] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5144.487332] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5145.335706] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5146.490249] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5147.950079] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5148.927971] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5149.878458] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5150.725621] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5151.791870] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5153.289073] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5154.176380] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5155.245521] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5156.022392] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5156.957686] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5157.940498] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5158.747190] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5159.518709] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5160.431959] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5161.604658] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5163.014868] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5164.068412] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5164.856778] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5165.788242] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5166.680211] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5167.445952] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5168.304254] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5169.340729] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5170.159377] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5171.095605] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5172.069416] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5172.978925] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5173.972128] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5174.835618] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5175.662810] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5176.454787] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5177.176803] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5178.015042] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5178.769242] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5179.590579] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5180.486434] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5181.440515] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5182.343454] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5183.244120] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5183.944984] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5184.732879] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5185.492815] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5186.263057] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5187.079526] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5187.907531] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5188.638084] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5189.502691] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5190.327892] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5191.205814] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5191.909950] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5192.686421] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5193.518291] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5194.211464] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5195.041655] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5195.819454] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5196.605224] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5197.436643] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5198.190720] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5198.977189] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5199.888767] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5200.711608] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5201.517588] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5202.428637] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5203.257038] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5204.047190] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5204.928560] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5205.695717] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5206.572345] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5207.314855] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5208.218178] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5208.993760] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5210.020235] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5210.816170] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5211.647119] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5212.530214] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5213.428515] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 5213.431574] Lustre: Skipped 138 previous similar messages [ 5213.452075] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5213.460546] Lustre: Skipped 138 previous similar messages [ 5213.464105] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 5213.468172] Lustre: Skipped 138 previous similar messages [ 5213.472387] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5213.482523] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 5213.487722] Lustre: Skipped 138 previous similar messages [ 5214.439835] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5215.362597] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5216.118348] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5216.889386] LustreError: 172068:0:(osp_precreate.c:655:osp_precreate_send()) lustre-OST0000-osc-MDT0000: can't precreate: rc = -5 [ 5216.897281] LustreError: 172068:0:(osp_precreate.c:1359:osp_precreate_thread()) lustre-OST0000-osc-MDT0000: cannot precreate objects: rc = -5 [ 5216.915174] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5217.779843] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5218.723714] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5219.800745] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5220.539036] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5221.318380] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5222.220614] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5223.300982] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5224.107086] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5224.962833] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5225.856460] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5226.714714] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5227.521537] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5228.452072] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5229.421057] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5230.213606] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5231.165932] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5232.076990] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5232.977197] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5233.804876] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5234.695363] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5235.583496] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5236.545476] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5237.212227] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5237.949340] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5238.759750] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5239.690629] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5240.938161] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5242.001218] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5242.983149] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5244.039812] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5245.113078] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5246.067419] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5246.922615] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5247.653613] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5248.479886] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5249.579801] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5250.439535] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5251.262321] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5252.150045] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5253.005238] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5253.884881] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5254.772393] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5255.821562] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5256.817306] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5257.766114] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5258.778643] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5259.862964] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5260.727379] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5261.458814] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5262.173789] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5263.004341] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5263.896093] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5264.835136] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5265.905444] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5266.604047] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5267.432295] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5268.254293] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5269.060991] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5270.022388] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5270.892435] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5271.540947] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5272.214554] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5272.941692] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5273.761915] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5274.538091] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5275.242315] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5276.052396] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5276.856343] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5277.657721] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5278.531187] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5279.498898] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5280.374865] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5281.245607] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5282.072640] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5282.844987] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5283.702293] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5284.574979] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5285.235499] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5286.024989] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5286.588891] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5287.365039] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5288.258835] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5289.115153] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5289.868543] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5290.654953] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5291.311475] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5292.008986] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5292.798331] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5293.536594] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5294.275814] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5295.104235] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5295.883543] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5296.717706] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5297.491677] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5298.216597] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5298.987914] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5299.780633] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5300.610063] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5301.323243] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5301.918335] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5302.600940] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5303.226051] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5303.975062] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5304.652070] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5305.454971] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5306.119604] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5306.804803] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5307.466277] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5308.215597] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5308.966704] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5309.863772] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5310.668926] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5311.426713] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5312.059970] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5312.764316] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5313.425365] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5314.103256] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5314.713297] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5315.325316] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5316.016832] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5316.825624] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5317.677443] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5318.419255] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5319.108533] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5320.004870] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5320.823704] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5321.694365] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5322.396583] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5323.097166] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5323.875271] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5324.806577] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5325.617398] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5326.591546] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5327.379875] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5328.288448] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5329.256348] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5330.290541] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5331.150964] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5332.021186] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5332.730325] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5333.510868] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5334.349438] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5335.162323] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5335.910087] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5336.984087] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5337.896795] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5338.859785] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5339.820430] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5340.645578] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5341.587462] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5342.408368] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5343.349831] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5344.272816] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5345.110642] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5346.083715] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5346.950919] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5347.970226] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5348.796682] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5349.559810] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5350.510957] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5351.394327] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5352.167870] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5352.973500] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5354.125582] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5355.165234] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5356.183559] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5357.100174] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5357.999487] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5358.871411] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5359.703153] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5360.538869] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5361.270278] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5362.035291] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5362.808486] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5363.583510] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5364.461168] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5365.264626] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5366.093168] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5367.002149] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5367.912900] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5368.689267] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5369.495492] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5370.252272] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5371.051688] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5371.949037] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5373.136457] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5374.156465] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5375.135313] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5375.922701] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5376.796701] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5377.822402] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5378.710158] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5379.517935] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5380.276360] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5381.125230] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5382.305981] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5383.234069] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5384.213650] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5385.014473] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5385.850336] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5386.746836] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5387.785707] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5388.940799] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5390.072093] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5391.123917] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5392.274728] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5393.295150] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5394.240943] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5395.123389] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5396.048971] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5397.101050] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5398.035295] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5399.113963] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5400.166260] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5401.065377] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5402.061938] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5403.148908] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5404.318658] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5405.391457] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5406.117592] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5406.873310] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5407.741450] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5408.506315] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5409.355709] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5410.311723] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5411.094494] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5412.275758] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5413.736403] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5414.565437] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5415.418114] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5416.358858] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5417.459531] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5418.423220] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5419.365299] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5420.325978] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5421.312523] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5422.274996] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5423.448135] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5424.392433] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5425.337216] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5427.169197] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5428.093924] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5429.003300] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5429.857656] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5431.025508] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5431.738984] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5432.773872] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5433.712704] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5434.626722] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5435.613895] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5436.726212] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5437.846256] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5438.865978] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5440.180993] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5441.064403] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5442.002865] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5443.064039] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5444.040499] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5445.604981] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5446.679463] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5447.735310] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5448.565981] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5449.382599] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5450.269879] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5451.156396] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5452.327455] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5453.392930] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5454.406525] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5455.132279] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5455.956945] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5456.750140] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5457.656014] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5458.556057] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5459.451313] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5460.529080] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5461.642136] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5462.704947] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5463.786272] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5464.707689] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5465.737912] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5466.681642] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5467.524466] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5468.245432] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5468.987610] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5469.696962] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 5469.702860] Lustre: Skipped 288 previous similar messages [ 5469.717647] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5469.731923] Lustre: Skipped 288 previous similar messages [ 5469.737770] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 5469.742653] Lustre: Skipped 288 previous similar messages [ 5469.747976] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5469.764647] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 5469.769292] Lustre: Skipped 288 previous similar messages [ 5470.553930] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5471.401341] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5472.265294] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5472.872204] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5473.686303] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5474.385916] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5475.152724] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5475.937856] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5476.651853] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5477.407248] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5478.290451] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5479.096066] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5479.777554] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5480.438655] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5481.160731] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5481.803858] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5482.539892] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5483.135112] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5483.719752] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5484.351284] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5484.988652] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5485.922780] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5486.779865] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5487.457254] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5488.314761] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5489.110218] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5490.065583] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5491.036453] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5491.817384] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5492.460036] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5493.367714] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5494.047603] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5494.975662] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5495.717605] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5496.347552] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5496.974780] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5497.762358] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5498.373613] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5499.304342] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5500.213759] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5501.031901] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5501.954292] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5502.917285] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5503.881768] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5504.826048] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5505.617068] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5506.461769] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5507.514842] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5508.464746] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5509.162598] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5509.813685] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5510.767309] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5511.741282] LustreError: 172068:0:(osp_precreate.c:655:osp_precreate_send()) lustre-OST0000-osc-MDT0000: can't precreate: rc = -5 [ 5511.753158] LustreError: 172068:0:(osp_precreate.c:1359:osp_precreate_thread()) lustre-OST0000-osc-MDT0000: cannot precreate objects: rc = -5 [ 5511.764569] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5512.872929] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5513.814743] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5514.886921] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5515.747283] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5516.605074] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5517.711194] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5518.684247] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5519.462977] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5520.367319] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5521.267018] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5522.114820] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5522.854860] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5523.603901] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5524.311828] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5525.106756] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5525.824298] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5526.525558] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5527.402314] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5528.095114] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5528.877345] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5529.576532] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5530.318988] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5530.971131] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5531.596945] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5532.159396] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5532.817057] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5533.433395] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5534.112676] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5534.780983] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5535.455555] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5536.074186] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5536.758705] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5537.431500] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5538.035701] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5538.625952] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5539.200238] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5539.736816] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5540.291684] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5540.862721] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5541.535124] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5542.160442] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5542.844500] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5543.455898] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5544.026790] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5544.506188] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5545.027180] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5545.632368] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5546.224397] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5546.839275] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5547.452431] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5548.073689] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5548.650788] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5549.301623] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5549.827935] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5550.394899] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5550.959468] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5551.576680] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5552.127848] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5552.862873] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5553.495205] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5554.144767] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5554.707383] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5555.437987] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5556.153378] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5556.803642] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5557.441101] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5558.101266] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5558.753223] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5559.520808] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5560.182873] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5560.927099] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5561.589719] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5562.238328] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5562.858876] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5563.564254] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5564.230733] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5564.967874] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5565.759280] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5566.522342] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5567.245889] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5568.019057] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5568.953256] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5569.888633] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5570.636066] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5571.418422] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5572.094860] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5572.732485] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5573.501992] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5574.421855] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5575.198455] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5576.169599] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5576.915971] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5577.664860] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5578.382550] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5579.149378] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5579.861972] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5580.650210] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5581.590972] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5582.452777] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5583.209703] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5584.072419] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5584.741654] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5585.356053] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5586.009184] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5586.939211] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5587.753539] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5588.649803] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5589.469744] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5590.323886] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5591.142603] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5591.968864] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5592.793407] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5593.489423] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5594.239613] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5594.960723] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5595.905348] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5596.749698] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5597.494432] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5598.405790] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5599.406392] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5600.266166] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5601.113833] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5601.981511] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5603.156571] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5604.197559] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5605.451831] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5606.379427] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5607.176996] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5607.968715] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5608.862353] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5609.585950] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5610.382686] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5611.194498] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5612.118302] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5613.070317] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5613.795647] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5614.847214] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5615.761987] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5616.700581] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5617.622306] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5618.680505] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5619.552588] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5620.372091] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5621.234886] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5622.035615] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5622.766492] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5623.564066] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5624.392515] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5625.455865] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5626.403798] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5627.415533] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5628.306600] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5629.235635] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5630.040313] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5630.715363] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5631.690881] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5632.490906] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5633.340867] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5634.168071] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5634.999669] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5635.762047] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5636.533112] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5637.182525] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5637.872580] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5638.737158] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5639.526736] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5640.216709] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5640.893333] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5641.759512] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5642.565774] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5643.353924] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5644.156721] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5644.930154] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5645.686978] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5646.323430] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5646.934727] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5647.552019] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5648.204555] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5648.910514] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5649.631419] LustreError: 172068:0:(osp_precreate.c:655:osp_precreate_send()) lustre-OST0000-osc-MDT0000: can't precreate: rc = -5 [ 5649.643514] LustreError: 172068:0:(osp_precreate.c:1359:osp_precreate_thread()) lustre-OST0000-osc-MDT0000: cannot precreate objects: rc = -5 [ 5649.645833] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5650.367331] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5651.088398] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5651.828189] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5652.490582] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5653.154130] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5654.047334] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5654.778174] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5655.782826] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5656.655443] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5657.344932] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5658.005975] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5658.661076] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5659.306778] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5660.010702] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5660.716654] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5661.411982] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5662.236942] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5663.085827] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5663.907893] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5664.645476] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5665.404487] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5666.117646] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5666.907814] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5667.728282] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5668.434754] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5669.281738] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5670.117855] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5670.843362] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5671.629223] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5672.405126] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5673.194653] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5673.961418] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5674.813534] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5675.705404] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5676.558862] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5677.347179] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5678.119763] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5678.898812] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5679.649651] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5680.416891] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5681.111971] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5681.800471] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5682.510032] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5683.330908] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5684.064352] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5684.788257] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5685.506765] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5686.197984] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5686.943180] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5687.678575] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5688.475172] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5689.376489] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5690.240137] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5691.217397] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5692.155715] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5692.983966] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5693.850326] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5694.817722] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5695.727168] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5696.544697] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5697.415416] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5698.453175] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5699.216201] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5700.081763] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5700.861405] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5701.798342] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5702.677765] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5703.540355] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5704.463050] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5705.399400] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5706.314281] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5707.389189] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5708.367651] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5709.327341] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5710.229381] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5711.119059] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5711.856162] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5712.608258] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5713.389823] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5714.205250] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5715.036174] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5715.958910] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5716.941340] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5717.791058] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5718.673353] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5719.463366] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5720.286159] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5721.184374] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5721.949152] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5722.728621] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5723.455967] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5724.333577] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5725.212817] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5726.128075] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5726.969877] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5727.808118] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5728.584665] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5729.387611] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5730.261452] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5731.127863] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5732.147990] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5733.027428] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5733.972618] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5734.966731] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5735.727113] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5736.572635] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5737.430199] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5738.435787] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5739.321839] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5740.377785] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5741.433061] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5742.394506] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5743.401656] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5744.295161] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5745.512526] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5746.332316] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5747.687665] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5748.501195] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5749.624638] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5750.868819] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5751.667656] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5752.628444] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5753.826304] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5754.818527] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5756.087285] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5757.093037] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5758.461836] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5760.048774] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5761.266763] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5762.590728] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5763.577799] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5764.832060] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5765.907883] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5766.930480] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5768.271317] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5769.232864] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5770.069026] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5770.872098] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5771.679570] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5772.552994] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5773.330516] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5774.192686] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5775.280260] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5776.526712] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5777.656789] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5778.738296] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5779.857782] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5780.740448] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5781.559961] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5782.273426] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5783.134759] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5783.971574] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5784.835264] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5785.655574] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5786.505746] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5787.317560] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5788.186542] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5789.158941] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5790.170553] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5791.157313] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5792.036211] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5792.913671] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5793.747866] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5794.571048] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5795.356369] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5796.092543] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5796.880805] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5797.819389] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5798.740454] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5799.536195] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5800.368438] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5801.399547] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5802.388155] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5803.206113] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5803.998912] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5804.830157] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5805.813585] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5806.563497] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5807.471912] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5808.306341] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5809.145860] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5810.032831] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5811.078342] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5811.996298] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5812.868366] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5813.769245] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5814.731559] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5815.461301] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6222.275035] LustreError: 173872:0:(ofd_dev.c:1783:ofd_create_hdl()) lustre-OST0000: unable to precreate: rc = -28 [ 6222.281930] LustreError: 172068:0:(osp_precreate.c:655:osp_precreate_send()) lustre-OST0000-osc-MDT0000: can't precreate: rc = -28 [ 6865.378734] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 6865.390801] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 6865.413091] Lustre: Skipped 422 previous similar messages [ 6865.434750] Lustre: Skipped 1 previous similar message [ 6870.244265] Lustre: server umount lustre-OST0000 complete [ 6875.621794] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 6880.739436] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 6880.756334] Lustre: Skipped 2 previous similar messages [ 6881.346975] Lustre: server umount lustre-MDT0000 complete [ 6885.605530] LustreError: 172784:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1787841333 with bad export cookie 15598389646486225180 [ 6885.607724] LustreError: MGC192.168.206.119@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 6885.619962] LustreError: 172784:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 3 previous similar messages [ 6886.113695] Lustre: server umount lustre-MDT0001 complete [ 6897.754764] Lustre: DEBUG MARKER: oleg619-server.virtnet: executing unload_modules_local [ 6900.627887] Key type lgssc unregistered [ 6900.952264] LNet: 227298:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 6900.959619] LNetError: 227298:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 6900.987338] LNet: Removed LNI 192.168.206.119@tcp [ 6901.753151] Key type .llcrypt unregistered [ 6901.756109] Key type ._llcrypt unregistered [ 6925.159259] Key type ._llcrypt registered [ 6925.161357] Key type .llcrypt registered [ 6925.253161] Lustre: DEBUG MARKER: oleg619-server.virtnet: executing load_modules_local [ 6926.204366] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 6926.225253] alg: No test for adler32 (adler32-zlib) [ 6927.239672] Lustre: Lustre: Build Version: 2.17.57_98_gaf12cc4 [ 6927.472313] LNet: Added LNI 192.168.206.119@tcp [8/256/0/180] [ 6929.161306] Key type lgssc registered [ 6930.213271] Lustre: Echo OBD driver; http://www.lustre.org/ [ 6937.240330] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 6937.278259] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 6938.654580] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 6942.092496] Lustre: DEBUG MARKER: oleg619-server.virtnet: executing set_default_debug -1 all [ 6948.839591] LustreError: 228710:0:(ldlm_lib.c:1228:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 6948.864174] LustreError: 228710:0:(ldlm_lib.c:1228:target_handle_connect()) Skipped 2 previous similar messages [ 6949.919429] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 6950.225724] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 6953.757803] Lustre: DEBUG MARKER: oleg619-server.virtnet: executing set_default_debug -1 all [ 6960.519515] Lustre: DEBUG MARKER: oleg619-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 6966.155745] Lustre: DEBUG MARKER: oleg619-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 6973.283221] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 6973.672355] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 6979.921914] Lustre: DEBUG MARKER: oleg619-server.virtnet: executing set_default_debug -1 all [ 6984.025956] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:49665 to 0x280000401:49696) [ 6987.093852] Lustre: DEBUG MARKER: oleg619-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 6991.719465] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 6993.449264] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 6993.454981] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 6993.460050] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6993.468532] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 6993.470391] LustreError: 228730:0:(osp_precreate.c:1268:osp_precreate_thread()) cfs_fail_timeout id 2107 sleeping for 20000ms [ 6996.452198] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 6996.462520] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 6996.479847] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 6999.023494] Lustre: lustre-OST0000: Not available for connect from 192.168.206.19@tcp (stopping) [ 6999.526166] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 7001.934802] Lustre: server umount lustre-OST0000 complete [ 7013.552874] LustreError: 228730:0:(osp_precreate.c:1268:osp_precreate_thread()) cfs_fail_timeout id 2107 awake [ 7035.939592] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 7036.236501] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 7039.346760] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7039.383855] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 7042.574626] Lustre: DEBUG MARKER: oleg619-server.virtnet: executing set_default_debug -1 all [ 7049.645344] LustreError: lustre-OST0000-osc-MDT0001: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7049.663757] Lustre: lustre-OST0000-osc-MDT0001: Connection restored to 0@lo (at 0@lo) [ 7049.812260] Lustre: DEBUG MARKER: oleg619-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 7051.349524] Lustre: DEBUG MARKER: osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid in FULL state after 0 sec [ 7057.004427] Lustre: DEBUG MARKER: oleg619-client.virtnet: executing wait_import_state (FULL|IDLE) osc.lustre-OST0000-osc-ffff8de7d8789800.ost_server_uuid 50 [ 7058.568385] Lustre: DEBUG MARKER: osc.lustre-OST0000-osc-ffff8de7d8789800.ost_server_uuid in FULL state after 0 sec [ 7064.033982] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 7064.050282] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 7064.077235] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 7064.088459] Lustre: Skipped 2 previous similar messages [ 7065.063769] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 7069.025443] Lustre: server umount lustre-OST0000 complete [ 7073.250422] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 7073.255575] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 7073.261598] Lustre: Skipped 1 previous similar message [ 7073.265519] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 7073.269101] Lustre: Skipped 2 previous similar messages [ 7079.325850] Lustre: server umount lustre-MDT0000 complete [ 7080.422803] LustreError: 228711:0:(ldlm_lib.c:1228:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 7080.458227] LustreError: 228711:0:(ldlm_lib.c:1228:target_handle_connect()) Skipped 1 previous similar message [ 7084.064170] LustreError: 228690:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1787841531 with bad export cookie 5355671481600951942 [ 7084.071966] LustreError: MGC192.168.206.119@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 7084.080349] LustreError: 228690:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 1 previous similar message [ 7084.884474] Lustre: server umount lustre-MDT0001 complete [ 7098.035458] Lustre: DEBUG MARKER: oleg619-server.virtnet: executing unload_modules_local [ 7100.959722] Key type lgssc unregistered [ 7101.329920] LNet: 233921:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 7101.338883] LNetError: 233921:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 7101.386752] LNet: Removed LNI 192.168.206.119@tcp [ 7102.334881] Key type .llcrypt unregistered [ 7102.337285] Key type ._llcrypt unregistered [ 7134.046345] Key type ._llcrypt registered [ 7134.049381] Key type .llcrypt registered [ 7134.171947] Lustre: DEBUG MARKER: oleg619-server.virtnet: executing load_modules_local [ 7135.432558] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 7135.455606] alg: No test for adler32 (adler32-zlib) [ 7136.508244] Lustre: Lustre: Build Version: 2.17.57_98_gaf12cc4 [ 7136.767920] LNet: Added LNI 192.168.206.119@tcp [8/256/0/180] [ 7138.440163] Key type lgssc registered [ 7139.629831] Lustre: Echo OBD driver; http://www.lustre.org/ [ 7148.656405] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 7148.691707] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 7150.180232] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 7154.662714] Lustre: DEBUG MARKER: oleg619-server.virtnet: executing set_default_debug -1 all [ 7159.277332] LustreError: 235647:0:(ldlm_lib.c:1228:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 7159.297259] LustreError: 235647:0:(ldlm_lib.c:1228:target_handle_connect()) Skipped 2 previous similar messages [ 7163.723751] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 7164.027364] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 7168.110464] Lustre: DEBUG MARKER: oleg619-server.virtnet: executing set_default_debug -1 all [ 7171.223844] Lustre: Modifying parameter general.debug_raw_pointers=Y in log params [ 7177.957520] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 7178.284060] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 7183.948709] Lustre: DEBUG MARKER: oleg619-server.virtnet: executing set_default_debug -1 all [ 7192.612810] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 7192.874438] Lustre: lustre-OST0001: new disk, initializing [ 7192.882387] Lustre: srv-lustre-OST0001: No data found on store. Initialize space. [ 7192.895761] Lustre: 237955:0:(osd_compat.c:1353:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 7193.037052] Lustre: lustre-OST0001: Imperative Recovery not enabled, recovery window 60-180 [ 7195.145014] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:49698 to 0x280000401:49728) [ 7199.515369] Lustre: DEBUG MARKER: oleg619-server.virtnet: executing set_default_debug -1 all [ 7202.944938] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x00000002c0000400-0x0000000300000400]:1:ost [ 7202.953509] Lustre: cli-lustre-OST0001-super: Allocated super-sequence [0x00000002c0000400-0x0000000300000400]:1:ost] [ 7203.153809] Lustre: lustre-OST0001-osc-MDT0000: update sequence from 0x100010000 to 0x2c0000401 [ 7211.081773] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 7220.151628] Lustre: Setting parameter general.lod.*.mdt_hash=crush in log params [ 7234.023826] Lustre: lustre-MDT0000-lwp-OST0000: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 7234.026134] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 7234.039281] Lustre: Skipped 3 previous similar messages [ 7234.047384] Lustre: Skipped 3 previous similar messages [ 7239.145064] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 7239.154744] Lustre: Skipped 3 previous similar messages [ 7240.099255] Lustre: server umount lustre-MDT0000 complete [ 7244.160784] LustreError: 238845:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1787841692 with bad export cookie 18312839306507934591 [ 7244.166053] LustreError: MGC192.168.206.119@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 7244.177503] LustreError: 238845:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 7244.259041] LustreError: 235644:0:(ldlm_lib.c:1228:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 7244.264301] Lustre: lustre-MDT0001-lwp-OST0001: Connection to lustre-MDT0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 7244.272145] Lustre: lustre-MDT0001: Not available for connect from 0@lo (stopping) [ 7244.287653] LustreError: 235644:0:(ldlm_lib.c:1228:target_handle_connect()) Skipped 1 previous similar message [ 7244.323182] Lustre: Skipped 1 previous similar message [ 7244.630732] Lustre: server umount lustre-MDT0001 complete [ 7250.807735] Lustre: server umount lustre-OST0000 complete [ 7254.995654] Lustre: server umount lustre-OST0001 complete [ 7276.283416] Lustre: DEBUG MARKER: oleg619-server.virtnet: executing load_modules_local [ 7287.120061] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 7287.521138] LustreError: 241068:0:(ldlm_lib.c:1228:target_handle_connect()) mylustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 7287.605769] Lustre: mylustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 7292.001765] Lustre: DEBUG MARKER: oleg619-server.virtnet: executing set_default_debug -1 all [ 7292.899965] LustreError: 241069:0:(ldlm_lib.c:1228:target_handle_connect()) mylustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 7298.020976] LustreError: 241068:0:(ldlm_lib.c:1228:target_handle_connect()) mylustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 7300.538974] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 7301.099797] Lustre: mylustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 7306.164489] Lustre: DEBUG MARKER: oleg619-server.virtnet: executing set_default_debug -1 all [ 7309.379986] Lustre: 242209:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify general/debug_raw_pointers=Y (mode = 0) failed: rc = -17 [ 7315.927526] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 7323.352995] Lustre: DEBUG MARKER: oleg619-server.virtnet: executing set_default_debug -1 all [ 7325.674125] LustreError: 242562:0:(ldlm_lib.c:1228:target_handle_connect()) mylustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 7325.703839] Lustre: mylustre-OST0000: new connection from mylustre-MDT0001-mdtlov (cleaning up unused objects from 0x280000400:99 to 0x280000400:129) [ 7332.772819] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 7332.871672] Lustre: mylustre-OST0000: new connection from mylustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:49698 to 0x280000401:49760) [ 7333.178383] Lustre: mylustre-OST0001: Imperative Recovery not enabled, recovery window 60-180 [ 7333.187690] Lustre: Skipped 1 previous similar message [ 7338.512415] Lustre: mylustre-OST0001: new connection from mylustre-MDT0001-mdtlov (cleaning up unused objects from 0x2c0000400:3 to 0x2c0000400:33) [ 7339.876914] Lustre: DEBUG MARKER: oleg619-server.virtnet: executing set_default_debug -1 all [ 7347.495717] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 7351.041681] Lustre: 244080:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify general/lod.*.mdt_hash=crush (mode = 0) failed: rc = -17 [ 7362.529730] LustreError: mylustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 7362.541533] Lustre: mylustre-MDT0000-osp-MDT0001: Connection to mylustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 7362.558592] Lustre: mylustre-MDT0000: Not available for connect from 0@lo (stopping) [ 7362.568766] Lustre: Skipped 1 previous similar message [ 7367.949244] Lustre: server umount mylustre-MDT0000 complete [ 7369.190205] LustreError: 241068:0:(ldlm_lib.c:1228:target_handle_connect()) mylustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 7369.214219] LustreError: 241068:0:(ldlm_lib.c:1228:target_handle_connect()) Skipped 5 previous similar messages [ 7371.956988] LustreError: 242587:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1787841819 with bad export cookie 18312839306507937454 [ 7371.962833] LustreError: MGC192.168.206.119@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 7371.969606] LustreError: 242587:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 7372.350982] Lustre: server umount mylustre-MDT0001 complete [ 7385.597077] Lustre: server umount mylustre-OST0000 complete [ 7400.726234] Lustre: server umount mylustre-OST0001 complete [ 7424.836926] Lustre: DEBUG MARKER: oleg619-server.virtnet: executing load_modules_local [ 7437.895714] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 7438.610856] LustreError: 246391:0:(ldlm_lib.c:1228:target_handle_connect()) tfs-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 7438.696581] Lustre: tfs-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 7443.189161] Lustre: DEBUG MARKER: oleg619-server.virtnet: executing set_default_debug -1 all [ 7452.509277] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 7458.374156] Lustre: DEBUG MARKER: oleg619-server.virtnet: executing set_default_debug -1 all [ 7461.223516] Lustre: 247532:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify general/debug_raw_pointers=Y (mode = 0) failed: rc = -17 [ 7467.884074] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 7473.321163] Lustre: DEBUG MARKER: oleg619-server.virtnet: executing set_default_debug -1 all [ 7477.488818] Lustre: tfs-OST0000: new connection from tfs-MDT0001-mdtlov (cleaning up unused objects from 0x280000400:99 to 0x280000400:161) [ 7480.399044] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 7483.663379] Lustre: tfs-OST0000: new connection from tfs-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:49698 to 0x280000401:49792) [ 7483.882312] Lustre: tfs-OST0001: new connection from tfs-MDT0001-mdtlov (cleaning up unused objects from 0x2c0000400:35 to 0x2c0000400:65) [ 7486.683286] Lustre: DEBUG MARKER: oleg619-server.virtnet: executing set_default_debug -1 all [ 7493.573277] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 7496.944401] Lustre: 249403:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify general/lod.*.mdt_hash=crush (mode = 0) failed: rc = -17 [ 7504.354711] LustreError: tfs-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 7504.363377] Lustre: tfs-MDT0000-osp-MDT0001: Connection to tfs-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 7504.368153] Lustre: tfs-MDT0000: Not available for connect from 0@lo (stopping) [ 7504.381343] Lustre: Skipped 4 previous similar messages [ 7504.383704] Lustre: Skipped 4 previous similar messages [ 7510.626446] Lustre: server umount tfs-MDT0000 complete [ 7514.600713] LustreError: 247640:0:(ldlm_lib.c:1228:target_handle_connect()) tfs-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 7514.664386] LustreError: 247640:0:(ldlm_lib.c:1228:target_handle_connect()) Skipped 6 previous similar messages [ 7515.022187] LustreError: 246368:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1787841962 with bad export cookie 18312839306507939631 [ 7515.030306] LustreError: MGC192.168.206.119@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 7515.047600] LustreError: 246368:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 7515.455175] Lustre: server umount tfs-MDT0001 complete [ 7530.960108] Lustre: server umount tfs-OST0000 complete [ 7544.498027] Lustre: server umount tfs-OST0001 complete [ 7569.736706] Lustre: DEBUG MARKER: oleg619-server.virtnet: executing load_modules_local [ 7580.690269] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 7581.193900] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 7581.202332] Lustre: Skipped 3 previous similar messages [ 7585.720601] Lustre: DEBUG MARKER: oleg619-server.virtnet: executing set_default_debug -1 all [ 7593.685069] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 7598.857333] Lustre: DEBUG MARKER: oleg619-server.virtnet: executing set_default_debug -1 all [ 7601.884600] Lustre: 252800:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify general/debug_raw_pointers=Y (mode = 0) failed: rc = -17 [ 7609.320814] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 7616.183584] Lustre: DEBUG MARKER: oleg619-server.virtnet: executing set_default_debug -1 all [ 7619.066296] Lustre: lustre-OST0000: new connection from lustre-MDT0001-mdtlov (cleaning up unused objects from 0x280000400:99 to 0x280000400:193) [ 7624.819456] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 7626.039158] Lustre: lustre-OST0001: new connection from lustre-MDT0001-mdtlov (cleaning up unused objects from 0x2c0000400:35 to 0x2c0000400:97) [ 7626.045066] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:49698 to 0x280000401:49824) [ 7631.063833] Lustre: DEBUG MARKER: oleg619-server.virtnet: executing set_default_debug -1 all [ 7638.559123] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 7641.766851] Lustre: 254670:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify general/lod.*.mdt_hash=crush (mode = 0) failed: rc = -17 [ 7648.318723] Lustre: DEBUG MARKER: == conf-sanity test 104a: Make sure user defined options are reflected in mount ========================================================== 10:48:13 (1787842093) [ 7665.635984] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 7665.647426] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 7665.659856] Lustre: Skipped 1 previous similar message [ 7665.672804] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 7665.684506] Lustre: Skipped 7 previous similar messages [ 7670.697218] Lustre: server umount lustre-MDT0000 complete [ 7672.293908] LustreError: 251657:0:(ldlm_lib.c:1228:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 7672.315727] LustreError: 251657:0:(ldlm_lib.c:1228:target_handle_connect()) Skipped 8 previous similar messages [ 7674.090766] LustreError: 251632:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1787842121 with bad export cookie 18312839306507941549 [ 7674.091869] LustreError: MGC192.168.206.119@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 7674.106107] LustreError: 251632:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 7677.421136] Lustre: lustre-MDT0001-lwp-OST0000: Connection to lustre-MDT0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 7677.434481] Lustre: Skipped 4 previous similar messages [ 7680.480414] Lustre: server umount lustre-MDT0001 complete [ 7690.534473] Lustre: server umount lustre-OST0000 complete [ 7700.306465] Lustre: server umount lustre-OST0001 complete [ 7706.615548] Lustre: DEBUG MARKER: oleg619-server.virtnet: executing set_hostid [ 7716.071982] Lustre: DEBUG MARKER: oleg619-server.virtnet: executing load_modules_local [ 7748.230522] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 7748.427217] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 7748.476613] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 7748.542846] Lustre: lustre-MDT0000: new disk, initializing [ 7748.632591] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 188-180 [ 7748.637134] Lustre: Skipped 3 previous similar messages [ 7748.646611] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 7752.716855] Lustre: DEBUG MARKER: oleg619-server.virtnet: executing set_default_debug -1 all [ 7762.896821] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 7763.024294] Lustre: 257853:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 7763.100272] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [ 7763.115149] Lustre: Skipped 1 previous similar message [ 7763.212376] Lustre: lustre-MDT0001: new disk, initializing [ 7763.468141] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [ 7763.492721] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 7767.799435] Lustre: DEBUG MARKER: oleg619-server.virtnet: executing set_default_debug -1 all [ 7778.430970] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 7778.704806] Lustre: lustre-OST0000: new disk, initializing [ 7778.710983] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 7778.719332] Lustre: 259432:0:(osd_compat.c:1353:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 7780.770376] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [ 7780.783782] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 7780.894040] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 7784.976632] Lustre: DEBUG MARKER: oleg619-server.virtnet: executing set_default_debug -1 all [ 7794.514885] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 7794.632329] Lustre: lustre-OST0001: new disk, initializing [ 7794.636384] Lustre: srv-lustre-OST0001: No data found on store. Initialize space. [ 7794.643808] Lustre: 260308:0:(osd_compat.c:1353:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 7796.042961] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x00000002c0000400-0x0000000300000400]:1:ost [ 7796.054744] Lustre: cli-lustre-OST0001-super: Allocated super-sequence [0x00000002c0000400-0x0000000300000400]:1:ost] [ 7796.164167] Lustre: lustre-OST0001-osc-MDT0000: update sequence from 0x100010000 to 0x2c0000401 [ 7800.101607] Lustre: DEBUG MARKER: oleg619-server.virtnet: executing set_default_debug -1 all [ 7810.397856] Lustre: DEBUG MARKER: == conf-sanity test 104b: Mount uses last flock argument ========================================================== 10:50:55 (1787842255) [ 7824.089410] Lustre: DEBUG MARKER: == conf-sanity test 105: check file creation for ro and rw bind mnt pt ========================================================== 10:51:09 (1787842269) [ 7842.292025] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 7842.303307] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 7842.332510] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 7842.359508] Lustre: Skipped 9 previous similar messages [ 7845.047228] Lustre: server umount lustre-OST0000 complete [ 7850.472781] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 7850.482325] LustreError: Skipped 1 previous similar message [ 7854.801676] Lustre: server umount lustre-MDT0000 complete [ 7858.001350] LustreError: 257846:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1787842305 with bad export cookie 18312839306507943327 [ 7858.010518] LustreError: 257846:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 2 previous similar messages [ 7858.013969] LustreError: MGC192.168.206.119@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 7864.476480] Lustre: server umount lustre-MDT0001 complete [ 7874.943977] Lustre: DEBUG MARKER: oleg619-server.virtnet: executing unload_modules_local [ 7951.176580] Lustre: server umount lustre-OST0001 complete [ 7957.574326] Lustre: DEBUG MARKER: oleg619-server.virtnet: executing set_hostid [ 7971.212286] Lustre: DEBUG MARKER: oleg619-server.virtnet: executing load_modules_local [ 8010.144772] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 8010.426597] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 8010.467881] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 8010.566487] Lustre: lustre-MDT0000: new disk, initializing [ 8010.638986] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 8010.644742] Lustre: Skipped 3 previous similar messages [ 8010.659921] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 8015.489770] Lustre: DEBUG MARKER: oleg619-server.virtnet: executing set_default_debug -1 all [ 8026.746899] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 8026.872989] Lustre: 265154:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 8026.966166] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [ 8026.971829] Lustre: Skipped 1 previous similar message [ 8027.040718] Lustre: lustre-MDT0001: new disk, initializing [ 8027.194581] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [ 8027.207711] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 8032.459934] Lustre: DEBUG MARKER: oleg619-server.virtnet: executing set_default_debug -1 all [ 8043.399272] Lustre: DEBUG MARKER: oleg619-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 8050.933282] Lustre: DEBUG MARKER: oleg619-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 8059.086275] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 8059.278559] Lustre: lustre-OST0000: new disk, initializing [ 8059.281254] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 8059.284776] Lustre: 267123:0:(osd_compat.c:1353:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 8060.945883] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [ 8060.969621] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 8061.096060] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 8065.710782] Lustre: DEBUG MARKER: oleg619-server.virtnet: executing set_default_debug -1 all [ 8075.757949] Lustre: DEBUG MARKER: oleg619-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 8086.500817] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 8086.505280] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 8086.513270] Lustre: Skipped 6 previous similar messages [ 8086.533430] Lustre: Skipped 7 previous similar messages [ 8088.371401] Lustre: server umount lustre-OST0000 complete [ 8093.667730] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 8098.537915] Lustre: server umount lustre-MDT0000 complete [ 8101.857317] LustreError: 266832:0:(ldlm_lib.c:1228:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 8101.865943] LustreError: 266832:0:(ldlm_lib.c:1228:target_handle_connect()) Skipped 14 previous similar messages [ 8101.958995] LustreError: 265145:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1787842549 with bad export cookie 18312839306507944797 [ 8101.958995] LustreError: 265146:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1787842549 with bad export cookie 18312839306507944797 [ 8101.966711] LustreError: MGC192.168.206.119@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 8101.967122] LustreError: 265146:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 2 previous similar messages [ 8101.989836] LustreError: 265145:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 1 previous similar message [ 8102.250220] Lustre: server umount lustre-MDT0001 complete [ 8113.381786] Lustre: DEBUG MARKER: oleg619-server.virtnet: executing unload_modules_local [ 8116.453954] Key type lgssc unregistered [ 8117.058107] LNet: 268885:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 8117.069793] LNetError: 268885:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 8117.118764] LNet: Removed LNI 192.168.206.119@tcp [ 8118.348139] Key type .llcrypt unregistered [ 8118.349663] Key type ._llcrypt unregistered [ 8150.927653] Key type ._llcrypt registered [ 8150.932807] Key type .llcrypt registered [ 8151.025854] Lustre: DEBUG MARKER: oleg619-server.virtnet: executing load_modules_local [ 8151.870210] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 8151.903483] alg: No test for adler32 (adler32-zlib) [ 8153.085880] Lustre: Lustre: Build Version: 2.17.57_98_gaf12cc4 [ 8153.273100] LNet: Added LNI 192.168.206.119@tcp [8/256/0/180] [ 8154.952731] Key type lgssc registered [ 8156.159682] Lustre: Echo OBD driver; http://www.lustre.org/ [ 8164.064873] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 8164.098586] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 8165.524076] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 8169.864289] Lustre: DEBUG MARKER: oleg619-server.virtnet: executing set_default_debug -1 all [ 8176.103903] LustreError: 270363:0:(ldlm_lib.c:1228:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 8176.123382] LustreError: 270363:0:(ldlm_lib.c:1228:target_handle_connect()) Skipped 2 previous similar messages [ 8179.419407] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 8180.009342] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 8184.768374] Lustre: DEBUG MARKER: oleg619-server.virtnet: executing set_default_debug -1 all [ 8192.672089] Lustre: DEBUG MARKER: oleg619-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 8198.484760] Lustre: DEBUG MARKER: oleg619-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 8205.801252] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 8206.365918] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 8210.513942] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:3 to 0x280000401:33) [ 8213.598755] Lustre: DEBUG MARKER: oleg619-server.virtnet: executing set_default_debug -1 all [ 8221.637817] Lustre: DEBUG MARKER: oleg619-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 8222.650581] Lustre: Setting parameter lustre-OST0000.ost.unknown_param=50 in log lustre-OST0000 [ 8225.645152] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 8225.649380] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 8225.676168] Lustre: Skipped 1 previous similar message [ 8230.862651] Lustre: server umount lustre-OST0000 complete [ 8236.001991] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 8236.029790] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 8236.038974] Lustre: Skipped 1 previous similar message [ 8241.122720] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 8241.131549] Lustre: Skipped 1 previous similar message [ 8241.733374] Lustre: server umount lustre-MDT0000 complete [ 8246.136584] LustreError: 271098:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1787842693 with bad export cookie 16683919256408694393 [ 8246.142681] LustreError: MGC192.168.206.119@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 8246.150124] LustreError: 271098:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 1 previous similar message [ 8246.879032] Lustre: server umount lustre-MDT0001 complete [ 8259.648226] Lustre: DEBUG MARKER: oleg619-server.virtnet: executing unload_modules_local [ 8262.438529] Key type lgssc unregistered [ 8262.776808] LNet: 273952:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 8262.784459] LNetError: 273952:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 8262.803421] LNet: Removed LNI 192.168.206.119@tcp [ 8263.588329] Key type .llcrypt unregistered [ 8263.590810] Key type ._llcrypt unregistered [ 8281.496351] Key type ._llcrypt registered [ 8281.503482] Key type .llcrypt registered [ 8281.636613] Lustre: DEBUG MARKER: oleg619-server.virtnet: executing load_modules_local [ 8282.862792] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 8282.900326] alg: No test for adler32 (adler32-zlib) [ 8283.982783] Lustre: Lustre: Build Version: 2.17.57_98_gaf12cc4 [ 8284.195994] LNet: Added LNI 192.168.206.119@tcp [8/256/0/180] [ 8285.872183] Key type lgssc registered [ 8287.231804] Lustre: Echo OBD driver; http://www.lustre.org/ [ 8296.699334] Lustre: lustre-OST0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 8296.712116] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 8312.290587] LustreError: Failed to get MGS log lustre-sptlrpc and no local copy. [ 8317.408569] LustreError: 275146:0:(mgc_request.c:233:do_config_log_add()) MGC192.168.206.119@tcp: failed processing log, type 4: rc = -110 [ 8344.097677] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 8351.079854] Lustre: DEBUG MARKER: oleg619-server.virtnet: executing set_default_debug -1 all [ 8359.058494] Lustre: DEBUG MARKER: oleg619-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 8361.223604] Lustre: server umount lustre-OST0000 complete [ 8377.127193] Lustre: DEBUG MARKER: oleg619-server.virtnet: executing unload_modules_local [ 8380.775398] Key type lgssc unregistered [ 8381.076698] LNet: 276781:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 8381.079589] LNetError: 276781:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 8381.104674] LNet: Removed LNI 192.168.206.119@tcp [ 8381.867375] Key type .llcrypt unregistered [ 8381.871178] Key type ._llcrypt unregistered [ 8424.406097] Key type ._llcrypt registered [ 8424.407881] Key type .llcrypt registered [ 8424.515795] Lustre: DEBUG MARKER: oleg619-server.virtnet: executing load_modules_local [ 8425.691367] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 8425.715635] alg: No test for adler32 (adler32-zlib) [ 8426.838606] Lustre: Lustre: Build Version: 2.17.57_98_gaf12cc4 [ 8427.076150] LNet: Added LNI 192.168.206.119@tcp [8/256/0/180] [ 8428.832350] Key type lgssc registered [ 8430.189086] Lustre: Echo OBD driver; http://www.lustre.org/ [ 8443.372362] loop0: detected capacity change from 0 to 204800000 [ 8443.595637] blk_update_request: operation not supported error, dev loop0, sector 399872 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [ 8443.620484] blk_update_request: operation not supported error, dev loop0, sector 208 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [ 8443.966436] blk_update_request: operation not supported error, dev loop0, sector 160232 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [ 8444.943533] loop0: detected capacity change from 0 to 204800000 [ 8445.096495] blk_update_request: operation not supported error, dev loop0, sector 399872 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [ 8445.113339] blk_update_request: operation not supported error, dev loop0, sector 208 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [ 8445.482513] blk_update_request: operation not supported error, dev loop0, sector 160232 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [ 8446.489926] loop0: detected capacity change from 0 to 204800000 [ 8446.704471] blk_update_request: operation not supported error, dev loop0, sector 399872 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [ 8446.718737] blk_update_request: operation not supported error, dev loop0, sector 8224 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [ 8446.843587] blk_update_request: operation not supported error, dev loop0, sector 58264 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [ 8447.923766] loop0: detected capacity change from 0 to 204800000 [ 8448.110117] blk_update_request: operation not supported error, dev loop0, sector 399872 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [ 8449.209420] loop0: detected capacity change from 0 to 204800000 [ 8449.236392] LDISKFS-fs (loop0): mounted filesystem with ordered data mode. Opts: (null) [ 8453.525954] loop0: detected capacity change from 0 to 204800000 [ 8453.544540] LDISKFS-fs (loop0): mounted filesystem with ordered data mode. Opts: (null) [ 8457.513855] loop0: detected capacity change from 0 to 204800000 [ 8457.568078] LDISKFS-fs (loop0): mounted filesystem with ordered data mode. Opts: (null) [ 8461.272895] loop0: detected capacity change from 0 to 204800000 [ 8461.322561] LDISKFS-fs (loop0): mounted filesystem with ordered data mode. Opts: (null) [ 8465.192344] loop0: detected capacity change from 0 to 204800000 [ 8465.283583] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 8465.330657] LDISKFS-fs (loop0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 8465.423440] Lustre: 280207:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0000: restore index 'fld' with [0x200000001:0x3:0x0]: rc = 0 [ 8465.430356] Lustre: 280207:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0000: restore index 'nodemap' with [0x200000003:0x3:0x0]: rc = 0 [ 8465.441193] Lustre: 280207:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0000: restore index '0x1010000-MDT0000' with [0x200000005:0x2:0x0]: rc = 0 [ 8465.456576] Lustre: 280207:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0000: restore index '0x1010000' with [0x200000003:0xb:0x0]: rc = 0 [ 8465.476823] Lustre: 280207:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0000: restore index '0x10000' with [0x200000003:0xa:0x0]: rc = 0 [ 8465.486040] Lustre: 280207:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0000: restore index '0x2010000' with [0x200000003:0xc:0x0]: rc = 0 [ 8465.509045] Lustre: 280207:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0000: restore index '0x10000-MDT0000' with [0x200000005:0x1:0x0]: rc = 0 [ 8465.534474] Lustre: 280207:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0000: restore index '0x2010000-MDT0000' with [0x200000005:0x3:0x0]: rc = 0 [ 8465.546454] Lustre: 280207:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0000: restore index '0x20000' with [0x200000006:0x20000:0x0]: rc = 0 [ 8465.565862] Lustre: 280207:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0000: restore index '0x2020000' with [0x200000006:0x2020000:0x0]: rc = 0 [ 8465.579924] Lustre: 280207:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0000: restore index '0x1020000' with [0x200000006:0x1020000:0x0]: rc = 0 [ 8465.596285] Lustre: 280207:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0000: restore index '0x1010000' with [0x200000006:0x1010000:0x0]: rc = 0 [ 8465.615982] Lustre: 280207:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0000: restore index '0x10000' with [0x200000006:0x10000:0x0]: rc = 0 [ 8465.625579] Lustre: 280207:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0000: restore index '0x2010000' with [0x200000006:0x2010000:0x0]: rc = 0 [ 8470.815063] Lustre: server umount MGS complete [ 8471.607962] loop0: detected capacity change from 0 to 204800000 [ 8471.689953] LDISKFS-fs (loop0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 8472.126681] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 8472.159948] Lustre: lustre-MDT0000: Aborting client recovery [ 8473.081056] loop1: detected capacity change from 0 to 204800000 [ 8473.252883] LDISKFS-fs (loop1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 8473.355551] Lustre: 280642:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0001: restore index 'fld' with [0x200000001:0x3:0x0]: rc = 0 [ 8473.377550] Lustre: 280642:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0001: restore index '0x10000-MDT0001' with [0x200000005:0x1:0x0]: rc = 0 [ 8473.391363] Lustre: 280642:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0001: restore index '0x2010000' with [0x200000003:0x6:0x0]: rc = 0 [ 8473.402277] Lustre: 280642:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0001: restore index '0x1010000' with [0x200000003:0x5:0x0]: rc = 0 [ 8473.410555] Lustre: 280642:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0001: restore index '0x1010000-MDT0001' with [0x200000005:0x2:0x0]: rc = 0 [ 8473.422689] Lustre: 280642:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0001: restore index '0x10000' with [0x200000003:0x4:0x0]: rc = 0 [ 8473.435201] Lustre: 280642:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0001: restore index '0x2010000-MDT0001' with [0x200000005:0x3:0x0]: rc = 0 [ 8473.474199] Lustre: Found index 1 for lustre-MDT0001, updating log [ 8473.647845] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 8473.698208] Lustre: lustre-MDT0001: Aborting client recovery [ 8474.544576] loop2: detected capacity change from 0 to 204800000 [ 8474.669409] LDISKFS-fs (loop2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 8474.731930] Lustre: 280720:0:(osd_scrub.c:1854:osd_index_restore()) lustre-OST0000: restore index 'fld' with [0x200000001:0x3:0x0]: rc = 0 [ 8474.742471] Lustre: 280720:0:(osd_scrub.c:1854:osd_index_restore()) lustre-OST0000: restore index '0x1020000' with [0x200000003:0x6:0x0]: rc = 0 [ 8474.756540] Lustre: 280720:0:(osd_scrub.c:1854:osd_index_restore()) lustre-OST0000: restore index '0x20000-OST0000' with [0x200000005:0x1:0x0]: rc = 0 [ 8474.763705] Lustre: 280720:0:(osd_scrub.c:1854:osd_index_restore()) lustre-OST0000: restore index '0x20000' with [0x200000003:0x5:0x0]: rc = 0 [ 8474.771653] Lustre: 280720:0:(osd_scrub.c:1854:osd_index_restore()) lustre-OST0000: restore index '0x1020000-OST0000' with [0x200000005:0x2:0x0]: rc = 0 [ 8474.782983] Lustre: 280720:0:(osd_scrub.c:1854:osd_index_restore()) lustre-OST0000: restore index '0x2020000-OST0000' with [0x200000005:0x3:0x0]: rc = 0 [ 8474.797539] Lustre: 280720:0:(osd_scrub.c:1854:osd_index_restore()) lustre-OST0000: restore index '0x2020000' with [0x200000003:0x7:0x0]: rc = 0 [ 8474.877923] Lustre: Found index 0 for lustre-OST0000, updating log [ 8474.907924] Lustre: 280738:0:(osd_compat.c:1353:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 8474.967205] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 8474.998607] Lustre: lustre-OST0000: Aborting recovery [ 8475.808803] loop3: detected capacity change from 0 to 204800000 [ 8475.949903] LDISKFS-fs (loop3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 8476.072388] Lustre: 280803:0:(osd_scrub.c:1854:osd_index_restore()) lustre-OST0001: restore index 'fld' with [0x200000001:0x3:0x0]: rc = 0 [ 8476.080523] Lustre: 280803:0:(osd_scrub.c:1854:osd_index_restore()) lustre-OST0001: restore index '0x2020000-OST0001' with [0x200000005:0x3:0x0]: rc = 0 [ 8476.087925] Lustre: 280803:0:(osd_scrub.c:1854:osd_index_restore()) lustre-OST0001: restore index '0x1020000' with [0x200000003:0x6:0x0]: rc = 0 [ 8476.095501] Lustre: 280803:0:(osd_scrub.c:1854:osd_index_restore()) lustre-OST0001: restore index '0x1020000-OST0001' with [0x200000005:0x2:0x0]: rc = 0 [ 8476.106193] Lustre: 280803:0:(osd_scrub.c:1854:osd_index_restore()) lustre-OST0001: restore index '0x2020000' with [0x200000003:0x7:0x0]: rc = 0 [ 8476.116614] Lustre: 280803:0:(osd_scrub.c:1854:osd_index_restore()) lustre-OST0001: restore index '0x20000-OST0001' with [0x200000005:0x1:0x0]: rc = 0 [ 8476.128661] Lustre: 280803:0:(osd_scrub.c:1854:osd_index_restore()) lustre-OST0001: restore index '0x20000' with [0x200000003:0x5:0x0]: rc = 0 [ 8476.152678] Lustre: Found index 1 for lustre-OST0001, updating log [ 8476.175623] Lustre: 280808:0:(osd_compat.c:1353:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 8476.240742] Lustre: lustre-OST0001: Aborting recovery [ 8481.264317] LustreError: 280586:0:(lod_dev.c:511:lod_sub_recovery_thread()) lustre-MDT0001-osp-MDT0000: get update log duration 10, retries 0, failed: rc = -108 [ 8482.277116] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x0:6 to 0x0:33) [ 8482.293309] Lustre: lustre-OST0001: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x0:6 to 0x0:33) [ 8482.306597] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000bd0 [ 8482.316457] Lustre: lustre-OST0001-osc-MDT0000: update sequence from 0x100010000 to 0x2c0000bd0 [ 8483.311094] Lustre: lustre-OST0001: new connection from lustre-MDT0001-mdtlov (cleaning up unused objects from 0x2c0000400:5 to 0x2c0000400:33) [ 8483.313771] Lustre: lustre-OST0000: new connection from lustre-MDT0001-mdtlov (cleaning up unused objects from 0x280000400:5 to 0x280000400:33) [ 8483.803209] LustreError: 280552:0:(mdt_identity.c:115:mdt_identity_do_upcall()) lustre-MDT0000: error invoking upcall /root/Work/Lustre/R5/lustre-release/lustre/tests/../utils/l_getidentity lustre-MDT0000 0: rc -2; check /proc/fs/lustre/mdt/lustre-MDT0000/identity_upcall, time 1465us: rc = -2 [ 8484.313031] LustreError: 280552:0:(mdt_identity.c:115:mdt_identity_do_upcall()) lustre-MDT0000: error invoking upcall /root/Work/Lustre/R5/lustre-release/lustre/tests/../utils/l_getidentity lustre-MDT0000 0: rc -2; check /proc/fs/lustre/mdt/lustre-MDT0000/identity_upcall, time 487us: rc = -2 [ 8484.328158] LustreError: 280552:0:(mdt_identity.c:115:mdt_identity_do_upcall()) Skipped 22 previous similar messages [ 8498.660352] Lustre: lustre-MDT0000-lwp-OST0000: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 8498.661201] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 8498.664494] Lustre: Skipped 3 previous similar messages [ 8498.675022] Lustre: Skipped 2 previous similar messages [ 8502.809258] Lustre: server umount lustre-MDT0000 complete [ 8503.732626] LustreError: 280536:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1787842951 with bad export cookie 15360666973107944813 [ 8503.747841] LustreError: MGC192.168.206.119@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 8503.752643] LustreError: 280536:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 1 previous similar message [ 8503.777681] LustreError: 280551:0:(ldlm_lib.c:1228:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 8503.785367] Lustre: lustre-MDT0001-lwp-OST0000: Connection to lustre-MDT0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 8503.795913] LustreError: 280551:0:(ldlm_lib.c:1228:target_handle_connect()) Skipped 3 previous similar messages [ 8503.810568] Lustre: lustre-MDT0001: Not available for connect from 0@lo (stopping) [ 8503.821968] Lustre: Skipped 2 previous similar messages [ 8508.898153] LustreError: 280550:0:(ldlm_lib.c:1228:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 8508.916770] Lustre: lustre-MDT0001: Not available for connect from 0@lo (stopping) [ 8508.943173] Lustre: Skipped 1 previous similar message [ 8514.019066] LustreError: 280550:0:(ldlm_lib.c:1228:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 8514.019739] Lustre: lustre-MDT0001: Not available for connect from 0@lo (stopping) [ 8514.044088] LustreError: 280550:0:(ldlm_lib.c:1228:target_handle_connect()) Skipped 2 previous similar messages [ 8514.095477] Lustre: Skipped 1 previous similar message [ 8518.112666] Lustre: lustre-MDT0001 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 8518.369995] Lustre: server umount lustre-MDT0001 complete [ 8525.572167] Lustre: server umount lustre-OST0000 complete [ 8541.152137] Lustre: lustre-OST0001 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 8541.485305] Lustre: server umount lustre-OST0001 complete [ 8550.288682] Lustre: DEBUG MARKER: == conf-sanity test 109a: test lctl clear_conf fsname ==== 11:03:15 (1787842995) [ 8580.896936] Lustre: DEBUG MARKER: oleg619-server.virtnet: executing set_hostid [ 8590.619106] Lustre: DEBUG MARKER: oleg619-server.virtnet: executing load_modules_local [ 8625.585727] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 8625.991882] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 8626.048875] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 8626.173382] Lustre: lustre-MDT0000: new disk, initializing [ 8626.327082] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 8626.335908] Lustre: Skipped 1 previous similar message [ 8626.360624] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 8630.760204] Lustre: DEBUG MARKER: oleg619-server.virtnet: executing set_default_debug -1 all [ 8641.963990] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 8642.106102] Lustre: 284877:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 8642.163816] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [ 8642.167420] Lustre: Skipped 1 previous similar message [ 8642.237322] Lustre: lustre-MDT0001: new disk, initializing [ 8642.364064] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 8642.412077] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [ 8642.437728] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 8648.098580] Lustre: DEBUG MARKER: oleg619-server.virtnet: executing set_default_debug -1 all [ 8659.153849] Lustre: DEBUG MARKER: oleg619-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 8665.005192] Lustre: DEBUG MARKER: oleg619-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 8671.789546] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 8672.205865] Lustre: lustre-OST0000: new disk, initializing [ 8672.212413] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 8672.224877] Lustre: 286847:0:(osd_compat.c:1353:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 8672.304703] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 8679.457674] Lustre: DEBUG MARKER: oleg619-server.virtnet: executing set_default_debug -1 all [ 8680.588408] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [ 8680.603734] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 8680.777059] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 8688.774430] Lustre: DEBUG MARKER: oleg619-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 8693.535973] Lustre: Setting parameter lustre-MDT0000.mdd.atime_diff=62 in log lustre-MDT0000 [ 8705.547837] Lustre: Modifying parameter lustre-MDT0000.mdd.atime_diff=63 in log lustre-MDT0000 [ 8712.940963] Lustre: Setting parameter lustre-client.llite.max_read_ahead_mb=32 in log lustre-client [ 8762.849915] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 8762.858901] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 8762.862724] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 8762.873937] Lustre: Skipped 2 previous similar messages [ 8762.882431] Lustre: Skipped 1 previous similar message [ 8766.666631] Lustre: server umount lustre-OST0000 complete [ 8770.533836] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 8770.540032] LustreError: Skipped 1 previous similar message [ 8770.547898] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 8773.090443] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 8773.099887] Lustre: Skipped 2 previous similar messages [ 8776.563799] Lustre: server umount lustre-MDT0000 complete [ 8778.213597] LustreError: 285687:0:(ldlm_lib.c:1228:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 8778.232384] LustreError: 285687:0:(ldlm_lib.c:1228:target_handle_connect()) Skipped 1 previous similar message [ 8780.216897] LustreError: 286846:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1787843228 with bad export cookie 15360666973107947529 [ 8780.223039] LustreError: MGC192.168.206.119@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 8780.228890] LustreError: 286846:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 8780.595154] Lustre: server umount lustre-MDT0001 complete [ 8789.162352] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 8793.385522] Lustre: DEBUG MARKER: oleg619-server.virtnet: executing set_default_debug -1 all [ 8798.267441] Lustre: server umount MGS complete [ 8808.982157] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 8809.272346] LustreError: 290807:0:(ldlm_lib.c:1228:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 8809.298409] LustreError: 290820:0:(qmt_pool.c:1639:qmt_pool_add_rem()) lustre-QMT0000: can't remove lustre-OST0000_UUID pool 'pool1': rc = -22 [ 8809.378564] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 8812.963285] Lustre: DEBUG MARKER: oleg619-server.virtnet: executing set_default_debug -1 all [ 8819.690756] LustreError: 290808:0:(ldlm_lib.c:1228:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 8819.705590] LustreError: 290808:0:(ldlm_lib.c:1228:target_handle_connect()) Skipped 1 previous similar message [ 8819.873380] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 8824.003732] Lustre: DEBUG MARKER: oleg619-server.virtnet: executing set_default_debug -1 all [ 8830.329583] Lustre: DEBUG MARKER: oleg619-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 8835.314687] Lustre: DEBUG MARKER: oleg619-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 8841.864105] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 8842.229519] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 8842.245412] Lustre: Skipped 1 previous similar message [ 8849.007312] Lustre: DEBUG MARKER: oleg619-server.virtnet: executing set_default_debug -1 all [ 8856.517622] Lustre: DEBUG MARKER: oleg619-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 8883.666634] Lustre: server umount lustre-OST0000 complete [ 8890.340840] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 8890.356562] Lustre: Skipped 2 previous similar messages [ 8890.360749] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 8890.371368] Lustre: Skipped 1 previous similar message [ 8897.621348] Lustre: server umount lustre-MDT0000 complete [ 8900.576929] LustreError: 292762:0:(ldlm_lib.c:1228:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 8900.595564] LustreError: 292762:0:(ldlm_lib.c:1228:target_handle_connect()) Skipped 1 previous similar message [ 8901.134265] LustreError: 290787:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1787843348 with bad export cookie 15360666973107949377 [ 8901.143522] LustreError: MGC192.168.206.119@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 8901.148482] LustreError: 290787:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 2 previous similar messages [ 8911.543830] Lustre: server umount lustre-MDT0001 complete [ 8923.799932] Lustre: DEBUG MARKER: oleg619-server.virtnet: executing unload_modules_local [ 8926.420325] Key type lgssc unregistered [ 8926.677505] LNet: 294753:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 8926.684831] LNetError: 294753:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 8926.699442] LNet: Removed LNI 192.168.206.119@tcp [ 8927.444780] Key type .llcrypt unregistered [ 8927.449164] Key type ._llcrypt unregistered [ 8963.800501] Key type ._llcrypt registered [ 8963.802851] Key type .llcrypt registered [ 8963.886443] Lustre: DEBUG MARKER: oleg619-server.virtnet: executing set_hostid [ 8977.428991] Lustre: DEBUG MARKER: oleg619-server.virtnet: executing load_modules_local [ 8978.562522] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 8978.595919] alg: No test for adler32 (adler32-zlib) [ 8979.577793] Lustre: Lustre: Build Version: 2.17.57_98_gaf12cc4 [ 8979.831919] LNet: Added LNI 192.168.206.119@tcp [8/256/0/180] [ 8981.560161] Key type lgssc registered [ 8982.482268] Lustre: Echo OBD driver; http://www.lustre.org/ [ 9012.002841] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 9012.046186] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 9013.227730] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 9013.249628] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 9013.312580] Lustre: lustre-MDT0000: new disk, initializing [ 9013.394067] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 9013.407683] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 9016.749373] Lustre: DEBUG MARKER: oleg619-server.virtnet: executing set_default_debug -1 all [ 9026.463971] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 9026.563424] Lustre: 298711:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 9026.629912] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [ 9026.636431] Lustre: Skipped 1 previous similar message [ 9026.745854] Lustre: lustre-MDT0001: new disk, initializing [ 9026.818970] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 9026.851936] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [ 9026.866153] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 9031.584490] Lustre: DEBUG MARKER: oleg619-server.virtnet: executing set_default_debug -1 all [ 9041.172608] Lustre: DEBUG MARKER: oleg619-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 9047.441963] Lustre: DEBUG MARKER: oleg619-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 9054.449788] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 9054.673345] Lustre: lustre-OST0000: new disk, initializing [ 9054.679055] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 9054.693101] Lustre: 300680:0:(osd_compat.c:1353:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 9054.784476] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 9060.479131] Lustre: DEBUG MARKER: oleg619-server.virtnet: executing set_default_debug -1 all [ 9061.464291] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [ 9061.482740] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 9061.587983] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 9070.313871] Lustre: DEBUG MARKER: oleg619-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 9077.353417] Lustre: Setting parameter lustre-MDT0000.mdd.atime_diff=62 in log lustre-MDT0000 [ 9088.087292] Lustre: Modifying parameter lustre-MDT0000.mdd.atime_diff=63 in log lustre-MDT0000 [ 9092.023479] Lustre: Setting parameter lustre-client.llite.max_read_ahead_mb=32 in log lustre-client [ 9138.659747] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 9138.660245] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 9138.667728] LustreError: Skipped 1 previous similar message [ 9138.669245] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 9138.707850] Lustre: Skipped 1 previous similar message [ 9143.806235] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 9143.812194] Lustre: Skipped 2 previous similar messages [ 9144.506453] Lustre: server umount lustre-OST0000 complete [ 9148.900683] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 9148.903848] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 9148.912918] Lustre: Skipped 1 previous similar message [ 9148.927368] Lustre: Skipped 1 previous similar message [ 9154.020233] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 9154.025113] Lustre: Skipped 1 previous similar message [ 9159.142486] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 9159.149927] Lustre: Skipped 1 previous similar message [ 9162.208139] Lustre: lustre-MDT0000 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 9162.369226] Lustre: server umount lustre-MDT0000 complete [ 9164.961375] LustreError: 301930:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1787843612 with bad export cookie 9684077651966413082 [ 9164.964040] LustreError: MGC192.168.206.119@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 9164.970854] LustreError: 301930:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 2 previous similar messages [ 9165.217388] Lustre: server umount lustre-MDT0001 complete [ 9171.256658] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 9174.662345] Lustre: DEBUG MARKER: oleg619-server.virtnet: executing set_default_debug -1 all [ 9178.648407] Lustre: server umount MGS complete [ 9188.686679] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 9189.093081] LustreError: 304500:0:(ldlm_lib.c:1228:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 9189.108625] LustreError: 304500:0:(ldlm_lib.c:1228:target_handle_connect()) Skipped 3 previous similar messages [ 9189.118750] LustreError: 304513:0:(qmt_pool.c:1639:qmt_pool_add_rem()) lustre-QMT0000: can't remove lustre-OST0000_UUID pool 'pool1': rc = -22 [ 9189.173373] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 9192.998408] Lustre: DEBUG MARKER: oleg619-server.virtnet: executing set_default_debug -1 all [ 9194.469239] LustreError: 304501:0:(ldlm_lib.c:1228:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 9198.563961] LustreError: 304500:0:(ldlm_lib.c:1228:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 9199.798170] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 9200.218781] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 9203.845193] Lustre: DEBUG MARKER: oleg619-server.virtnet: executing set_default_debug -1 all [ 9210.816509] Lustre: DEBUG MARKER: oleg619-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 9216.577172] Lustre: DEBUG MARKER: oleg619-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 9223.869775] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 9224.207469] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 9230.163846] Lustre: DEBUG MARKER: oleg619-server.virtnet: executing set_default_debug -1 all [ 9237.597340] Lustre: DEBUG MARKER: oleg619-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 9266.147620] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 9266.158993] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 9266.200220] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 9269.217789] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 9269.231470] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 9271.538723] Lustre: server umount lustre-OST0000 complete [ 9276.897712] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 9276.905871] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 9284.577762] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 9284.597561] Lustre: Skipped 6 previous similar messages [ 9290.208587] Lustre: lustre-MDT0000 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 9290.457635] Lustre: server umount lustre-MDT0000 complete [ 9294.207563] LustreError: 304480:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1787843742 with bad export cookie 9684077651966414916 [ 9294.211703] LustreError: MGC192.168.206.119@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 9294.217266] LustreError: 304480:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 2 previous similar messages [ 9294.679753] Lustre: server umount lustre-MDT0001 complete [ 9304.535610] Lustre: DEBUG MARKER: oleg619-server.virtnet: executing unload_modules_local [ 9307.394065] Key type lgssc unregistered [ 9307.660795] LNet: 308449:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 9307.670700] LNetError: 308449:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 9307.686803] LNet: Removed LNI 192.168.206.119@tcp [ 9308.454308] Key type .llcrypt unregistered [ 9308.456985] Key type ._llcrypt unregistered [ 9332.575101] Key type ._llcrypt registered [ 9332.577381] Key type .llcrypt registered [ 9332.724985] Lustre: DEBUG MARKER: oleg619-server.virtnet: executing set_hostid [ 9347.043768] Lustre: DEBUG MARKER: oleg619-server.virtnet: executing load_modules_local [ 9348.177923] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 9348.217811] alg: No test for adler32 (adler32-zlib) [ 9349.265917] Lustre: Lustre: Build Version: 2.17.57_98_gaf12cc4 [ 9349.464821] LNet: Added LNI 192.168.206.119@tcp [8/256/0/180] [ 9351.168150] Key type lgssc registered [ 9352.375578] Lustre: Echo OBD driver; http://www.lustre.org/ [ 9377.859982] Lustre: DEBUG MARKER: oleg619-server.virtnet: executing load_modules_local [ 9393.755488] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 9393.790130] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 9395.070579] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 9395.110268] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 9395.237434] Lustre: lustre-MDT0000: new disk, initializing [ 9395.326393] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 9395.347686] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 9399.922946] Lustre: DEBUG MARKER: oleg619-server.virtnet: executing set_default_debug -1 all [ 9410.805547] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 9411.190358] Lustre: lustre-OST0000: new disk, initializing [ 9411.201471] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 9411.210840] Lustre: Skipped 1 previous similar message [ 9411.215812] Lustre: 312502:0:(osd_compat.c:1353:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 9411.327978] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 9416.939088] Lustre: DEBUG MARKER: oleg619-server.virtnet: executing set_default_debug -1 all [ 9418.841313] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:0:ost [ 9418.852582] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:0:ost] [ 9418.882846] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x240000400 [10268.641625] Lustre: lustre-MDT0000-lwp-OST0000: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [10268.659665] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [10271.292441] Lustre: server umount lustre-MDT0000 complete [10274.170819] LustreError: 311687:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1787844722 with bad export cookie 10342849841150894941 [10274.173554] LustreError: MGC192.168.206.119@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [10274.179178] LustreError: 311687:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 2 previous similar messages [10274.315472] Lustre: server umount lustre-OST0000 complete [10298.499592] Lustre: DEBUG MARKER: oleg619-server.virtnet: executing set_hostid [10308.127825] Lustre: DEBUG MARKER: oleg619-server.virtnet: executing load_modules_local [10340.392963] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [10340.609400] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [10340.642983] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [10340.729520] Lustre: lustre-MDT0000: new disk, initializing [10340.797685] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [10340.811986] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [10344.527469] Lustre: DEBUG MARKER: oleg619-server.virtnet: executing set_default_debug -1 all [10354.223095] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [10354.310774] Lustre: 316837:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [10354.350515] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [10354.358263] Lustre: Skipped 1 previous similar message [10354.441476] Lustre: lustre-MDT0001: new disk, initializing [10354.523460] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [10354.579455] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [10354.595268] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [10357.970543] Lustre: DEBUG MARKER: oleg619-server.virtnet: executing set_default_debug -1 all [10366.899845] Lustre: DEBUG MARKER: oleg619-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [10373.638026] Lustre: DEBUG MARKER: oleg619-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [10380.770712] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [10380.990678] Lustre: lustre-OST0000: new disk, initializing [10380.993758] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [10380.999333] Lustre: 318803:0:(osd_compat.c:1353:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [10381.072772] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [10386.587589] Lustre: DEBUG MARKER: oleg619-server.virtnet: executing set_default_debug -1 all [10388.525984] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [10388.535726] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [10388.641860] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [10395.394056] Lustre: DEBUG MARKER: oleg619-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [10400.978548] Lustre: DEBUG MARKER: oleg619-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 [10401.236920] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [10408.524328] Lustre: DEBUG MARKER: oleg619-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid 50 [10408.905334] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [10414.054205] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [10414.065908] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [10414.088933] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [10414.571037] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [10418.297294] Lustre: server umount lustre-OST0000 complete [10424.805724] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [10424.816155] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [10424.817043] Lustre: Skipped 2 previous similar messages [10424.832257] Lustre: Skipped 2 previous similar messages [10429.341556] Lustre: server umount lustre-MDT0000 complete [10433.597143] LustreError: 316829:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1787844881 with bad export cookie 10342849841153835760 [10433.604029] LustreError: MGC192.168.206.119@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [10433.609251] LustreError: 316829:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [10434.067717] Lustre: server umount lustre-MDT0001 complete [10437.862409] Lustre: DEBUG MARKER: SKIP: conf-sanity test_111 skipping SLOW test 111 [10439.718408] Lustre: DEBUG MARKER: == conf-sanity test 112a: mount OST with no_create option ========================================================== 11:34:44 (1787844884) [10446.927649] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [10447.230362] LustreError: 321115:0:(ldlm_lib.c:1228:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [10447.247425] LustreError: 321115:0:(ldlm_lib.c:1228:target_handle_connect()) Skipped 3 previous similar messages [10447.288962] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [10451.558033] Lustre: DEBUG MARKER: oleg619-server.virtnet: executing set_default_debug -1 all [10452.455046] LustreError: 321116:0:(ldlm_lib.c:1228:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [10457.573288] LustreError: 321115:0:(ldlm_lib.c:1228:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [10458.860228] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [10459.161474] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [10463.500904] Lustre: DEBUG MARKER: oleg619-server.virtnet: executing set_default_debug -1 all [10470.976776] Lustre: DEBUG MARKER: oleg619-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [10476.377560] Lustre: DEBUG MARKER: oleg619-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [10483.557064] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [10483.758667] LustreError: 322936:0:(ldlm_lib.c:1228:target_handle_connect()) lustre-OST0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [10483.930084] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [10489.422817] Lustre: DEBUG MARKER: oleg619-server.virtnet: executing set_default_debug -1 all [10496.111685] Lustre: DEBUG MARKER: oleg619-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [10502.572936] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [10502.729661] Lustre: lustre-OST0001: new disk, initializing [10502.732827] Lustre: srv-lustre-OST0001: No data found on store. Initialize space. [10502.738368] Lustre: 323951:0:(osd_compat.c:1353:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [10502.824472] Lustre: lustre-OST0001: Imperative Recovery not enabled, recovery window 60-180 [10507.709283] Lustre: DEBUG MARKER: oleg619-server.virtnet: executing set_default_debug -1 all [10512.505888] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x00000002c0000400-0x0000000300000400]:1:ost [10512.516497] Lustre: cli-lustre-OST0001-super: Allocated super-sequence [0x00000002c0000400-0x0000000300000400]:1:ost] [10512.571582] LustreError: 324527:0:(osp_precreate.c:974:osp_precreate_cleanup_orphans()) lustre-OST0001-osc-MDT0001: cannot cleanup orphans: rc = -1 [10513.648048] LustreError: 324527:0:(osp_precreate.c:974:osp_precreate_cleanup_orphans()) lustre-OST0001-osc-MDT0001: cannot cleanup orphans: rc = -1 [10513.658595] LustreError: 324527:0:(osp_precreate.c:974:osp_precreate_cleanup_orphans()) Skipped 1 previous similar message [10514.724168] LustreError: 324527:0:(osp_precreate.c:974:osp_precreate_cleanup_orphans()) lustre-OST0001-osc-MDT0001: cannot cleanup orphans: rc = -1 [10514.737447] LustreError: 324527:0:(osp_precreate.c:974:osp_precreate_cleanup_orphans()) Skipped 2 previous similar messages [10516.436165] Lustre: DEBUG MARKER: oleg619-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 [10516.609640] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [10516.769955] LustreError: 324533:0:(osp_precreate.c:974:osp_precreate_cleanup_orphans()) lustre-OST0001-osc-MDT0000: cannot cleanup orphans: rc = -1 [10516.784249] LustreError: 324533:0:(osp_precreate.c:974:osp_precreate_cleanup_orphans()) Skipped 3 previous similar messages [10520.875555] LustreError: 324527:0:(osp_precreate.c:974:osp_precreate_cleanup_orphans()) lustre-OST0001-osc-MDT0001: cannot cleanup orphans: rc = -1 [10520.884351] LustreError: 324527:0:(osp_precreate.c:974:osp_precreate_cleanup_orphans()) Skipped 6 previous similar messages [10521.737989] Lustre: DEBUG MARKER: oleg619-client.virtnet: executing wait_import_state (FULL|IDLE) osc.lustre-OST0000-osc-ffff8de7c6e09800.ost_server_uuid 50 [10523.153799] Lustre: DEBUG MARKER: osc.lustre-OST0000-osc-ffff8de7c6e09800.ost_server_uuid in FULL state after 0 sec [10528.546501] Lustre: DEBUG MARKER: oleg619-client.virtnet: executing wait_import_state (FULL|IDLE) osc.lustre-OST0001-osc-ffff8de7c6e09800.ost_server_uuid 50 [10529.148888] LustreError: 324533:0:(osp_precreate.c:974:osp_precreate_cleanup_orphans()) lustre-OST0001-osc-MDT0000: cannot cleanup orphans: rc = -1 [10529.165056] LustreError: 324533:0:(osp_precreate.c:974:osp_precreate_cleanup_orphans()) Skipped 15 previous similar messages [10530.189545] Lustre: DEBUG MARKER: osc.lustre-OST0001-osc-ffff8de7c6e09800.ost_server_uuid in FULL state after 0 sec [10532.300585] Lustre: lustre-OST0001-osc-MDT0000: update sequence from 0x100010000 to 0x2c0000401 [10547.681890] LustreError: lustre-OST0001-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [10547.692147] Lustre: lustre-OST0001-osc-MDT0000: Connection to lustre-OST0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [10547.700626] LustreError: Skipped 1 previous similar message [10547.705351] Lustre: lustre-OST0001: Not available for connect from 0@lo (stopping) [10547.711719] Lustre: Skipped 1 previous similar message [10547.731824] Lustre: Skipped 1 previous similar message [10551.591034] Lustre: server umount lustre-OST0001 complete [10553.295049] LustreError: 322938:0:(ldlm_lib.c:1228:target_handle_connect()) lustre-OST0001: not available for connect from 192.168.206.19@tcp (no target). If you are running an HA pair check that the target is mounted on the other server. [10558.945221] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [10558.947045] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [10558.959152] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [10558.960723] Lustre: Skipped 1 previous similar message [10558.972682] Lustre: Skipped 4 previous similar messages [10563.437394] Lustre: server umount lustre-OST0000 complete [10569.188200] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [10569.195148] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [10569.201219] Lustre: Skipped 1 previous similar message [10574.305918] LustreError: 321115:0:(ldlm_lib.c:1228:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [10574.321780] LustreError: 321115:0:(ldlm_lib.c:1228:target_handle_connect()) Skipped 5 previous similar messages [10575.490332] Lustre: server umount lustre-MDT0000 complete [10579.019338] LustreError: 321095:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1787845026 with bad export cookie 10342849841153836642 [10579.020527] LustreError: MGC192.168.206.119@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [10579.037094] LustreError: 321095:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 1 previous similar message [10579.326614] Lustre: server umount lustre-MDT0001 complete [10589.551269] Lustre: DEBUG MARKER: oleg619-server.virtnet: executing unload_modules_local [10592.651325] Key type lgssc unregistered [10592.949330] LNet: 326643:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [10592.965496] LNetError: 326643:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [10592.986782] LNet: Removed LNI 192.168.206.119@tcp [10593.614898] Key type .llcrypt unregistered [10593.616896] Key type ._llcrypt unregistered [10618.216507] Key type ._llcrypt registered [10618.221381] Key type .llcrypt registered [10618.327578] Lustre: DEBUG MARKER: oleg619-server.virtnet: executing load_modules_local [10619.106508] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [10619.164477] alg: No test for adler32 (adler32-zlib) [10620.261520] Lustre: Lustre: Build Version: 2.17.57_98_gaf12cc4 [10620.491513] LNet: Added LNI 192.168.206.119@tcp [8/256/0/180] [10622.144741] Key type lgssc registered [10623.254156] Lustre: Echo OBD driver; http://www.lustre.org/ [10631.729203] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [10631.746833] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [10633.216645] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [10637.932464] Lustre: DEBUG MARKER: oleg619-server.virtnet: executing set_default_debug -1 all [10643.430537] LustreError: 328067:0:(ldlm_lib.c:1228:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [10643.454953] LustreError: 328067:0:(ldlm_lib.c:1228:target_handle_connect()) Skipped 2 previous similar messages [10645.664076] Lustre: DEBUG MARKER: oleg619-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [10648.552168] LustreError: 328068:0:(ldlm_lib.c:1228:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [10653.071718] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [10653.536957] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [10658.572196] Lustre: DEBUG MARKER: oleg619-server.virtnet: executing set_default_debug -1 all [10669.001475] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [10669.308183] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [10675.172388] Lustre: DEBUG MARKER: oleg619-server.virtnet: executing set_default_debug -1 all [10678.510405] LustreError: 329749:0:(ldlm_lib.c:1228:target_handle_connect()) lustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [10678.535651] LustreError: 329749:0:(ldlm_lib.c:1228:target_handle_connect()) Skipped 1 previous similar message [10678.536426] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:36 to 0x280000401:65) [10682.324141] Lustre: DEBUG MARKER: oleg619-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [10683.880816] LustreError: 330262:0:(ldlm_lib.c:1228:target_handle_connect()) lustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [10683.903087] LustreError: 330262:0:(ldlm_lib.c:1228:target_handle_connect()) Skipped 1 previous similar message [10688.999718] LustreError: 329749:0:(ldlm_lib.c:1228:target_handle_connect()) lustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [10689.024342] LustreError: 329749:0:(ldlm_lib.c:1228:target_handle_connect()) Skipped 1 previous similar message [10689.100127] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [10689.318206] Lustre: lustre-OST0001: Imperative Recovery not enabled, recovery window 60-180 [10694.677709] Lustre: lustre-OST0001: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x2c0000401:3 to 0x2c0000401:33) [10696.074204] Lustre: DEBUG MARKER: oleg619-server.virtnet: executing set_default_debug -1 all [10703.962751] Lustre: DEBUG MARKER: oleg619-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0001-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [10711.254976] Lustre: DEBUG MARKER: oleg619-server.virtnet: executing wait_import_state (FULL|IDLE) os[cp].lustre-OST0001-osc-MDT0001.ost_server_uuid 50 [10711.488695] Lustre: DEBUG MARKER: os[cp].lustre-OST0001-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [10817.509636] LustreError: lustre-OST0001-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [10817.529570] Lustre: lustre-OST0001-osc-MDT0001: Connection to lustre-OST0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [10817.557832] Lustre: lustre-OST0001: Not available for connect from 0@lo (stopping) [10817.567452] Lustre: Skipped 1 previous similar message [10820.535519] Lustre: server umount lustre-OST0001 complete [10822.629958] LustreError: 330263:0:(ldlm_lib.c:1228:target_handle_connect()) lustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [10822.642864] LustreError: 330263:0:(ldlm_lib.c:1228:target_handle_connect()) Skipped 1 previous similar message [10832.354729] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [10832.372879] LustreError: Skipped 1 previous similar message [10832.381683] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [10832.401762] Lustre: Skipped 1 previous similar message [10832.420579] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [10832.436520] Lustre: Skipped 1 previous similar message [10833.573963] Lustre: server umount lustre-OST0000 complete [10837.990803] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [10837.996180] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [10838.006296] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [10838.007020] LustreError: Skipped 1 previous similar message [10838.044983] Lustre: Skipped 2 previous similar messages [10843.106030] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [10843.127233] Lustre: Skipped 2 previous similar messages [10843.723307] Lustre: server umount lustre-MDT0000 complete [10847.604511] LustreError: 328048:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1787845295 with bad export cookie 11253076955306779094 [10847.612545] LustreError: MGC192.168.206.119@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [10847.619161] LustreError: 328048:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [10848.009897] Lustre: server umount lustre-MDT0001 complete [10859.558911] Lustre: DEBUG MARKER: oleg619-server.virtnet: executing unload_modules_local [10863.146615] Key type lgssc unregistered [10863.461857] LNet: 333014:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [10863.474864] LNetError: 333014:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [10863.502865] LNet: Removed LNI 192.168.206.119@tcp [10864.488483] Key type .llcrypt unregistered [10864.490989] Key type ._llcrypt unregistered [10908.921785] Key type ._llcrypt registered [10908.927871] Key type .llcrypt registered [10909.045737] Lustre: DEBUG MARKER: oleg619-server.virtnet: executing load_modules_local [10909.848055] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [10909.892874] alg: No test for adler32 (adler32-zlib) [10911.026504] Lustre: Lustre: Build Version: 2.17.57_98_gaf12cc4 [10911.303541] LNet: Added LNI 192.168.206.119@tcp [8/256/0/180] [10912.960170] Key type lgssc registered [10914.107500] Lustre: Echo OBD driver; http://www.lustre.org/ [10923.872179] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [10923.913995] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [10925.790419] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [10932.758489] Lustre: DEBUG MARKER: oleg619-server.virtnet: executing set_default_debug -1 all [10935.273900] LustreError: 335297:0:(ldlm_lib.c:1228:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [10935.321948] LustreError: 335297:0:(ldlm_lib.c:1228:target_handle_connect()) Skipped 2 previous similar messages [10940.389879] LustreError: 335298:0:(ldlm_lib.c:1228:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [10945.504750] LustreError: 335297:0:(ldlm_lib.c:1228:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [10946.689613] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [10947.411314] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [10952.052713] Lustre: DEBUG MARKER: oleg619-server.virtnet: executing set_default_debug -1 all [10956.280862] Lustre: Modifying parameter general.debug_raw_pointers=Y in log params [10966.349304] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [10967.020706] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [10971.134148] LustreError: 336791:0:(ldlm_lib.c:1228:target_handle_connect()) lustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [10971.163400] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:36 to 0x280000401:97) [10973.637947] Lustre: DEBUG MARKER: oleg619-server.virtnet: executing set_default_debug -1 all [10977.268589] LustreError: 336790:0:(ldlm_lib.c:1228:target_handle_connect()) lustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [10977.298945] LustreError: 336790:0:(ldlm_lib.c:1228:target_handle_connect()) Skipped 3 previous similar messages [10982.362303] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [10982.662076] Lustre: lustre-OST0001: Imperative Recovery not enabled, recovery window 60-180 [10988.034917] Lustre: lustre-OST0001: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x2c0000401:3 to 0x2c0000401:65) [10988.697892] Lustre: DEBUG MARKER: oleg619-server.virtnet: executing set_default_debug -1 all [10997.020075] Lustre: DEBUG MARKER: Using TIMEOUT=20 [11001.188978] Lustre: Setting parameter general.lod.*.mdt_hash=crush in log params [11011.558988] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [11011.572525] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [11011.576056] Lustre: Skipped 3 previous similar messages [11011.592141] Lustre: Skipped 3 previous similar messages [11016.673827] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [11016.684510] Lustre: Skipped 2 previous similar messages [11017.866930] Lustre: server umount lustre-MDT0000 complete [11021.443613] LustreError: 335280:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1787845469 with bad export cookie 13258563218666946026 [11021.461066] LustreError: MGC192.168.206.119@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [11021.463145] LustreError: 335280:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [11021.848765] Lustre: server umount lustre-MDT0001 complete [11035.636769] Lustre: server umount lustre-OST0000 complete [11037.153677] Lustre: 334563:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1787845469/real 1787845469] req@ffff9b981fda03c0 x1874691735223040/t0(0) o400->lustre-MDT0001-lwp-OST0001@0@lo:12/10 lens 224/224 e 0 to 1 dl 1787845485 ref 1 fl Rpc:XNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 [11037.192634] Lustre: lustre-MDT0001-lwp-OST0001: Connection to lustre-MDT0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [11039.140423] Lustre: server umount lustre-OST0001 complete [11055.838483] Lustre: DEBUG MARKER: oleg619-server.virtnet: executing load_modules_local [11064.494463] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [11064.863075] LustreError: 340561:0:(ldlm_lib.c:1228:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [11064.888297] LustreError: 340561:0:(ldlm_lib.c:1228:target_handle_connect()) Skipped 2 previous similar messages [11064.968920] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [11068.546508] Lustre: DEBUG MARKER: oleg619-server.virtnet: executing set_default_debug -1 all [11076.315413] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [11076.562763] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [11081.642843] Lustre: DEBUG MARKER: oleg619-server.virtnet: executing set_default_debug -1 all [11084.573498] Lustre: 341703:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify general/debug_raw_pointers=Y (mode = 0) failed: rc = -17 [11091.438876] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [11097.566596] Lustre: DEBUG MARKER: oleg619-server.virtnet: executing set_default_debug -1 all [11102.121186] LustreError: 342056:0:(ldlm_lib.c:1228:target_handle_connect()) lustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [11102.148703] LustreError: 342056:0:(ldlm_lib.c:1228:target_handle_connect()) Skipped 2 previous similar messages [11105.619513] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [11105.799488] Lustre: lustre-OST0001: Imperative Recovery not enabled, recovery window 60-180 [11105.803936] Lustre: Skipped 1 previous similar message [11109.890890] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:36 to 0x280000401:129) [11109.898816] Lustre: lustre-OST0001: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x2c0000401:3 to 0x2c0000401:97) [11111.453862] Lustre: DEBUG MARKER: oleg619-server.virtnet: executing set_default_debug -1 all [11118.586334] Lustre: DEBUG MARKER: Using TIMEOUT=20 [11122.078448] Lustre: 343573:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify general/lod.*.mdt_hash=crush (mode = 0) failed: rc = -17 [11127.777307] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [11127.782483] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [11127.792272] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [11130.340682] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [11130.341343] Lustre: lustre-MDT0000-lwp-OST0000: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [11130.351323] Lustre: Skipped 2 previous similar messages [11130.363751] Lustre: Skipped 2 previous similar messages [11132.033270] Lustre: server umount lustre-MDT0000 complete [11135.464522] LustreError: 340556:0:(ldlm_lib.c:1228:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [11135.474507] LustreError: 340556:0:(ldlm_lib.c:1228:target_handle_connect()) Skipped 3 previous similar messages [11135.569660] LustreError: 342886:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1787845583 with bad export cookie 13258563218666947720 [11135.572389] LustreError: MGC192.168.206.119@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [11135.578804] LustreError: 342886:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 2 previous similar messages [11135.942479] Lustre: server umount lustre-MDT0001 complete [11149.873780] Lustre: server umount lustre-OST0000 complete [11162.557925] Lustre: server umount lustre-OST0001 complete [11166.154339] Lustre: DEBUG MARKER: SKIP: conf-sanity test_114 skipping SLOW test 114 [11167.911301] Lustre: DEBUG MARKER: SKIP: conf-sanity test_115 skipping excluded test 115 [11169.900925] Lustre: DEBUG MARKER: == conf-sanity test 116: big size MDT support ============ 11:46:54 (1787845614) [11190.216981] Lustre: DEBUG MARKER: oleg619-server.virtnet: executing load_modules_local [11198.794529] loop0: detected capacity change from 0 to 1099511627776 [11199.446146] SGI XFS with ACLs, security attributes, quota, no debug enabled [11199.452214] *** VALIDATE xfs *** [11199.461735] XFS (loop0): Mounting V5 Filesystem [11199.480985] XFS (loop0): Ending clean mount [11200.669800] loop1: detected capacity change from 0 to 18691697672192 [11307.968079] Lustre: DEBUG MARKER: oleg619-server.virtnet: executing set_hostid [11321.254964] Lustre: DEBUG MARKER: oleg619-server.virtnet: executing load_modules_local [11363.379282] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [11363.826247] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [11363.894534] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [11364.038757] Lustre: lustre-MDT0000: new disk, initializing [11364.191211] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [11364.225653] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [11370.334028] Lustre: DEBUG MARKER: oleg619-server.virtnet: executing set_default_debug -1 all [11383.179175] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [11383.357519] Lustre: 348712:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [11383.422513] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [11383.426617] Lustre: Skipped 1 previous similar message [11383.548623] Lustre: lustre-MDT0001: new disk, initializing [11383.722913] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [11383.740043] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [11388.707610] Lustre: DEBUG MARKER: oleg619-server.virtnet: executing set_default_debug -1 all [11398.917401] Lustre: DEBUG MARKER: oleg619-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [11404.732414] Lustre: DEBUG MARKER: oleg619-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [11411.972111] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [11412.299556] Lustre: lustre-OST0000: new disk, initializing [11412.303844] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [11412.313936] Lustre: 350680:0:(osd_compat.c:1353:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [11413.459732] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [11413.468038] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [11413.580137] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [11418.728791] Lustre: DEBUG MARKER: oleg619-server.virtnet: executing set_default_debug -1 all [11428.910623] Lustre: DEBUG MARKER: oleg619-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [11433.675707] Lustre: DEBUG MARKER: oleg619-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 [11433.953566] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [11438.854551] Lustre: DEBUG MARKER: oleg619-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid 50 [11439.035889] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [11444.205838] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [11444.216640] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [11444.219236] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [11444.233113] Lustre: Skipped 1 previous similar message [11444.247544] LustreError: Skipped 1 previous similar message [11447.388829] Lustre: server umount lustre-OST0000 complete [11454.434156] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [11454.437647] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [11454.448612] Lustre: Skipped 1 previous similar message [11454.458104] Lustre: Skipped 2 previous similar messages [11458.092406] Lustre: server umount lustre-MDT0000 complete [11459.553911] LustreError: 348724:0:(ldlm_lib.c:1228:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [11459.580194] LustreError: 348724:0:(ldlm_lib.c:1228:target_handle_connect()) Skipped 1 previous similar message [11462.429215] LustreError: 349518:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1787845910 with bad export cookie 13258563218666949281 [11462.434083] LustreError: MGC192.168.206.119@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [11462.447410] LustreError: 349518:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 5 previous similar messages [11463.425094] Lustre: server umount lustre-MDT0001 complete [11473.166830] XFS (loop0): Unmounting Filesystem [11479.213425] Lustre: DEBUG MARKER: == conf-sanity test 117: lctl get_param return errors properly ========================================================== 11:52:04 (1787845924) [11486.757852] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [11487.507189] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [11487.518949] Lustre: Skipped 2 previous similar messages [11492.768260] Lustre: DEBUG MARKER: oleg619-server.virtnet: executing set_default_debug -1 all [11502.325912] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [11507.219782] Lustre: DEBUG MARKER: oleg619-server.virtnet: executing set_default_debug -1 all [11515.013602] Lustre: DEBUG MARKER: oleg619-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [11521.649522] Lustre: DEBUG MARKER: oleg619-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [11529.906392] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [11537.370102] Lustre: DEBUG MARKER: oleg619-server.virtnet: executing set_default_debug -1 all [11546.338482] Lustre: DEBUG MARKER: oleg619-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [11562.977046] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [11562.984460] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [11562.997519] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [11563.010076] Lustre: Skipped 1 previous similar message [11565.376970] Lustre: server umount lustre-OST0000 complete [11569.637459] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [11569.650391] LustreError: Skipped 1 previous similar message [11575.786314] Lustre: server umount lustre-MDT0000 complete [11580.014713] LustreError: 353220:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1787846027 with bad export cookie 13258563218666950156 [11580.016519] LustreError: MGC192.168.206.119@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [11580.027185] LustreError: 353220:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 2 previous similar messages [11580.510133] Lustre: server umount lustre-MDT0001 complete [11593.382918] Lustre: DEBUG MARKER: oleg619-server.virtnet: executing unload_modules_local [11596.413895] Key type lgssc unregistered [11596.819893] LNet: 356875:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [11596.826275] LNetError: 356875:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [11596.847951] LNet: Removed LNI 192.168.206.119@tcp [11597.953299] Key type .llcrypt unregistered [11597.955111] Key type ._llcrypt unregistered [11627.816478] Key type ._llcrypt registered [11627.818815] Key type .llcrypt registered [11627.935765] Lustre: DEBUG MARKER: oleg619-server.virtnet: executing load_modules_local [11628.760724] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [11628.912107] alg: No test for adler32 (adler32-zlib) [11630.015240] Lustre: Lustre: Build Version: 2.17.57_98_gaf12cc4 [11630.448747] LNet: Added LNI 192.168.206.119@tcp [8/256/0/180] [11632.216688] Key type lgssc registered [11633.312210] Lustre: Echo OBD driver; http://www.lustre.org/ [11641.962150] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [11642.028986] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [11643.552298] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [11649.162570] Lustre: DEBUG MARKER: oleg619-server.virtnet: executing set_default_debug -1 all [11654.117120] LustreError: 358455:0:(ldlm_lib.c:1228:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [11654.137666] LustreError: 358455:0:(ldlm_lib.c:1228:target_handle_connect()) Skipped 2 previous similar messages [11659.042463] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [11659.233528] Lustre: lustre-MDT0001: Not available for connect from 0@lo (not set up) [11659.489040] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [11664.195537] Lustre: DEBUG MARKER: oleg619-server.virtnet: executing set_default_debug -1 all [11673.272371] Lustre: DEBUG MARKER: oleg619-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [11680.745788] Lustre: DEBUG MARKER: oleg619-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [11689.808468] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [11690.265249] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [11697.155428] Lustre: DEBUG MARKER: oleg619-server.virtnet: executing set_default_debug -1 all [11705.266230] Lustre: DEBUG MARKER: oleg619-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [11726.319173] LustreError: lustre-MDT0001-osp-MDT0000: operation mds_statfs to node 0@lo failed: rc = -107 [11726.333141] Lustre: lustre-MDT0001-osp-MDT0000: Connection to lustre-MDT0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [11726.359750] Lustre: lustre-MDT0001: Not available for connect from 0@lo (stopping) [11727.850847] Lustre: lustre-MDT0001: Not available for connect from 192.168.206.19@tcp (stopping) [11729.387150] Lustre: lustre-MDT0001-lwp-OST0000: Connection to lustre-MDT0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [11731.427051] Lustre: server umount lustre-MDT0001 complete [11732.939797] LustreError: 358449:0:(ldlm_lib.c:1228:target_handle_connect()) lustre-MDT0001: not available for connect from 192.168.206.19@tcp (no target). If you are running an HA pair check that the target is mounted on the other server. [11734.505893] LustreError: 358450:0:(ldlm_lib.c:1228:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [11734.533634] LustreError: 358450:0:(ldlm_lib.c:1228:target_handle_connect()) Skipped 1 previous similar message [11738.067549] LustreError: 359678:0:(ldlm_lib.c:1228:target_handle_connect()) lustre-MDT0001: not available for connect from 192.168.206.19@tcp (no target). If you are running an HA pair check that the target is mounted on the other server. [11743.179560] LustreError: 359678:0:(ldlm_lib.c:1228:target_handle_connect()) lustre-MDT0001: not available for connect from 192.168.206.19@tcp (no target). If you are running an HA pair check that the target is mounted on the other server. [11743.190731] LustreError: 359678:0:(ldlm_lib.c:1228:target_handle_connect()) Skipped 2 previous similar messages [11743.778862] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [11743.898176] Lustre: MGS: Regenerating lustre-MDT0001 log by user request: rc = 0 [11743.910475] Lustre: Found index 1 for lustre-MDT0001, updating log [11743.943442] Lustre: 358441:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [11744.195333] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [11749.238416] Lustre: DEBUG MARKER: oleg619-server.virtnet: executing set_default_debug -1 all [11749.370886] Lustre: lustre-MDT0001-lwp-OST0000: Connection restored to 0@lo (at 0@lo) [11751.935943] LustreError: 362048:0:(obd_mount.c:234:lustre_start_simple()) lustre-MDT0001-lwp-OST0000 attach error -17 [11751.953247] LustreError: 362048:0:(tgt_mount.c:675:lustre_lwp_setup()) lustre-MDT0001-lwp-OST0000: setup up failed: rc -17 [11751.977751] LustreError: 358447:0:(mgc_request.c:614:do_requeue()) failed processing log: -17 [11752.094262] LustreError: 362056:0:(obd_config.c:2102:class_config_llog_handler()) MGC192.168.206.119@tcp: cfg command failed: rc = -17 [11752.111738] Lustre: cmd=cf001 0:lustre-MDT0001-osp-MDT0000 1:osp 2:lustre-MDT0000-mdtlov_UUID [11752.111738] [11763.682064] Lustre: lustre-MDT0001: Not available for connect from 192.168.206.19@tcp (stopping) [11763.691617] Lustre: Skipped 2 previous similar messages [11764.706845] LustreError: lustre-MDT0001-osp-MDT0000: operation mds_statfs to node 0@lo failed: rc = -107 [11764.720620] Lustre: lustre-MDT0001-osp-MDT0000: Connection to lustre-MDT0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [11764.729867] Lustre: Skipped 1 previous similar message [11766.400776] Lustre: server umount lustre-MDT0001 complete [11768.786168] LustreError: 358449:0:(ldlm_lib.c:1228:target_handle_connect()) lustre-MDT0001: not available for connect from 192.168.206.19@tcp (no target). If you are running an HA pair check that the target is mounted on the other server. [11778.140753] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [11778.403237] Lustre: MGS: Regenerating lustre-MDT0001 log by user request: rc = 0 [11778.412828] Lustre: Found index 1 for lustre-MDT0001, updating log [11778.475877] Lustre: 358444:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [11778.950135] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [11784.187574] Lustre: lustre-MDT0001-lwp-OST0000: Connection restored to 0@lo (at 0@lo) [11785.039781] Lustre: DEBUG MARKER: oleg619-server.virtnet: executing set_default_debug -1 all [11785.745985] LustreError: 363023:0:(obd_mount.c:234:lustre_start_simple()) lustre-MDT0001-lwp-OST0000 attach error -17 [11785.755410] LustreError: 363023:0:(tgt_mount.c:675:lustre_lwp_setup()) lustre-MDT0001-lwp-OST0000: setup up failed: rc -17 [11785.778950] LustreError: 358447:0:(mgc_request.c:614:do_requeue()) failed processing log: -17 [11785.786587] LustreError: 358447:0:(mgc_request.c:614:do_requeue()) Skipped 1 previous similar message [11785.955471] LustreError: 363031:0:(obd_config.c:761:class_setup()) osp: device 8 already setup: rc = -17 [11785.964844] LustreError: 363031:0:(obd_config.c:866:class_detach()) OBD device 8 still set up [11785.970964] LustreError: 363031:0:(obd_config.c:2102:class_config_llog_handler()) MGC192.168.206.119@tcp: cfg command failed: rc = -17 [11785.983670] Lustre: cmd=cf003 0:lustre-MDT0001-osp-MDT0000 1:lustre-MDT0001_UUID 2:192.168.206.119@tcp [11785.983670] [12095.458720] Lustre: lustre-MDT0001-lwp-OST0000: Connection to lustre-MDT0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [12095.485965] Lustre: lustre-MDT0001: Not available for connect from 0@lo (stopping) [12095.496350] Lustre: Skipped 3 previous similar messages [12098.382335] Lustre: server umount lustre-MDT0001 complete [12100.585153] LustreError: 358454:0:(ldlm_lib.c:1228:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [12100.609822] LustreError: 358454:0:(ldlm_lib.c:1228:target_handle_connect()) Skipped 6 previous similar messages [12108.651372] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [12108.746857] Lustre: MGS: Regenerating lustre-MDT0001 log by user request: rc = 0 [12108.755301] Lustre: Found index 1 for lustre-MDT0001, updating log [12108.795897] Lustre: 358444:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [12109.117653] Lustre: lustre-MDT0001: Imperative Recovery enabled, recovery window shrunk from 60-180 down to 60-180 [12113.613979] Lustre: DEBUG MARKER: oleg619-server.virtnet: executing set_default_debug -1 all [12114.410684] Lustre: lustre-MDT0001-lwp-OST0000: Connection restored to 0@lo (at 0@lo) [12118.523939] LustreError: 371649:0:(obd_mount.c:234:lustre_start_simple()) lustre-MDT0001-lwp-OST0000 attach error -17 [12118.533560] LustreError: 371649:0:(tgt_mount.c:675:lustre_lwp_setup()) lustre-MDT0001-lwp-OST0000: setup up failed: rc -17 [12118.547917] LustreError: 358447:0:(mgc_request.c:614:do_requeue()) failed processing log: -17 [12118.557849] LustreError: 358447:0:(mgc_request.c:614:do_requeue()) Skipped 1 previous similar message [12118.636931] LustreError: 371652:0:(lod_lov.c:208:lod_add_device()) lustre-MDT0001-osp-MDT0000: cannot connect to next dev lustre-MDT0001_UUID (-114) [12118.653888] LustreError: 371652:0:(obd_config.c:2102:class_config_llog_handler()) MGC192.168.206.119@tcp: cfg command failed: rc = -114 [12118.665622] Lustre: cmd=cf014 0:lustre-MDT0000-mdtlov 1:lustre-MDT0001_UUID 2:1 3:1 [12118.665622] [12436.966156] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [12436.972812] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [12436.978315] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [12436.991404] Lustre: Skipped 3 previous similar messages [12437.021092] Lustre: Skipped 3 previous similar messages [12441.612948] Lustre: server umount lustre-MDT0000 complete [12442.083899] LustreError: 358454:0:(ldlm_lib.c:1228:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [12442.105394] LustreError: 358454:0:(ldlm_lib.c:1228:target_handle_connect()) Skipped 6 previous similar messages [12445.414619] LustreError: 361661:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1787846893 with bad export cookie 7244448120617988736 [12445.421409] LustreError: MGC192.168.206.119@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [12445.432084] LustreError: 361661:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [12447.202410] Lustre: lustre-MDT0001-lwp-OST0000: Connection to lustre-MDT0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [12451.983577] Lustre: server umount lustre-MDT0001 complete [12456.594516] Lustre: server umount lustre-OST0000 complete [12466.206620] Lustre: DEBUG MARKER: oleg619-server.virtnet: executing set_hostid [12477.351519] Lustre: DEBUG MARKER: oleg619-server.virtnet: executing load_modules_local [12515.835654] Lustre: DEBUG MARKER: == conf-sanity test 120: cross-target rename should not create bad symlinks ========================================================== 12:09:20 (1787846960) [12522.358028] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [12522.576964] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [12522.619879] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [12522.699564] Lustre: lustre-MDT0000: new disk, initializing [12522.763823] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [12522.783754] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [12527.302701] Lustre: DEBUG MARKER: oleg619-server.virtnet: executing set_default_debug -1 all [12538.008672] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [12538.158053] Lustre: 381920:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [12538.263622] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [12538.268152] Lustre: Skipped 1 previous similar message [12538.355956] Lustre: lustre-MDT0001: new disk, initializing [12538.488309] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [12538.528480] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [12538.539524] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [12543.178435] Lustre: DEBUG MARKER: oleg619-server.virtnet: executing set_default_debug -1 all [12552.430321] Lustre: DEBUG MARKER: oleg619-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [12559.130778] Lustre: DEBUG MARKER: oleg619-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [12568.412381] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [12568.888477] Lustre: lustre-OST0000: new disk, initializing [12568.893764] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [12568.910946] Lustre: 383886:0:(osd_compat.c:1353:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [12569.057755] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [12575.934621] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [12575.946245] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [12576.148386] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [12577.758611] Lustre: DEBUG MARKER: oleg619-server.virtnet: executing set_default_debug -1 all [12587.619114] Lustre: DEBUG MARKER: oleg619-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [12596.706268] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [12596.711057] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [12596.734654] Lustre: Skipped 2 previous similar messages [12596.763860] Lustre: Skipped 4 previous similar messages [12598.660799] Lustre: server umount lustre-MDT0000 complete [12601.826107] LustreError: 381931:0:(ldlm_lib.c:1228:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [12601.842918] LustreError: 381931:0:(ldlm_lib.c:1228:target_handle_connect()) Skipped 4 previous similar messages [12602.853192] LustreError: 382725:0:(osp_object.c:618:osp_attr_get()) lustre-MDT0000-osp-MDT0001: osp_attr_get update error [0x200000401:0x2:0x0]: rc = -5 [12602.855431] LustreError: 381913:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1787847050 with bad export cookie 7244448120617990689 [12602.862132] LustreError: MGC192.168.206.119@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [12602.913811] LustreError: 381913:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [12603.349849] Lustre: server umount lustre-MDT0001 complete [12616.542397] Lustre: server umount lustre-OST0000 complete [12648.580656] Lustre: DEBUG MARKER: oleg619-server.virtnet: executing set_hostid [12657.520369] Lustre: DEBUG MARKER: oleg619-server.virtnet: executing load_modules_local [12698.380200] Lustre: DEBUG MARKER: == conf-sanity test complete, duration 12434 sec ========= 12:12:22 (1787847142) [12700.654783] Lustre: DEBUG MARKER: === conf-sanity: start cleanup 12:12:25 (1787847145) === [12704.840823] Lustre: DEBUG MARKER: === conf-sanity: finish cleanup 12:12:29 (1787847149) === [12747.483489] Lustre: DEBUG MARKER: oleg619-server.virtnet: executing unload_modules_local [12750.290902] Key type lgssc unregistered [12750.597170] LNet: 390258:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [12750.604345] LNetError: 390258:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [12750.615915] LNet: Removed LNI 192.168.206.119@tcp [12751.477063] Key type .llcrypt unregistered [12751.483365] Key type ._llcrypt unregistered