[ 0.000000] Linux version 4.18.0rh8.10-debug (green@maintenance) (gcc version 8.5.0 20210514 (Red Hat 8.5.0-26) (GCC)) #2 SMP Mon Jul 14 01:24:22 EDT 2025 [ 0.000000] Command line: rd.shell root=nbd:192.168.200.253:rocky8.10:ext4:ro:-p,-b4096 ro crashkernel=256M panic=1 nomodeset ipmtu=9000 ip=dhcp rd.neednet=1 init_on_free=off mitigations=off console=ttyS1,115200 audit=0 [ 0.000000] x86/fpu: Supporting XSAVE feature 0x001: 'x87 floating point registers' [ 0.000000] x86/fpu: Supporting XSAVE feature 0x002: 'SSE registers' [ 0.000000] x86/fpu: Supporting XSAVE feature 0x004: 'AVX registers' [ 0.000000] x86/fpu: xstate_offset[2]: 576, xstate_sizes[2]: 256 [ 0.000000] x86/fpu: Enabled xstate features 0x7, context size is 832 bytes, using 'standard' format. [ 0.000000] signal: max sigframe size: 1776 [ 0.000000] BIOS-provided physical RAM map: [ 0.000000] BIOS-e820: [mem 0x0000000000000000-0x000000000009fbff] usable [ 0.000000] BIOS-e820: [mem 0x000000000009fc00-0x000000000009ffff] reserved [ 0.000000] BIOS-e820: [mem 0x00000000000f0000-0x00000000000fffff] reserved [ 0.000000] BIOS-e820: [mem 0x0000000000100000-0x00000000bffcdfff] usable [ 0.000000] BIOS-e820: [mem 0x00000000bffce000-0x00000000bfffffff] reserved [ 0.000000] BIOS-e820: [mem 0x00000000feffc000-0x00000000feffffff] reserved [ 0.000000] BIOS-e820: [mem 0x00000000fffc0000-0x00000000ffffffff] reserved [ 0.000000] BIOS-e820: [mem 0x0000000100000000-0x0000000146dfffff] usable [ 0.000000] NX (Execute Disable) protection: active [ 0.000000] SMBIOS 2.8 present. [ 0.000000] DMI: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.17.0-10.fc44 06/10/2025 [ 0.000000] Hypervisor detected: KVM [ 0.000000] kvm-clock: Using msrs 4b564d01 and 4b564d00 [ 0.000000] kvm-clock: using sched offset of 499996733 cycles [ 0.000000] clocksource: kvm-clock: mask: 0xffffffffffffffff max_cycles: 0x1cd42e4dffb, max_idle_ns: 881590591483 ns [ 0.000000] tsc: Detected 2399.998 MHz processor [ 0.000000] last_pfn = 0x146e00 max_arch_pfn = 0x400000000 [ 0.000000] x86/PAT: Configuration [0-7]: WB WC UC- UC WB WP UC- WT [ 0.000000] last_pfn = 0xbffce max_arch_pfn = 0x400000000 [ 0.000000] found SMP MP-table at [mem 0x000f54b0-0x000f54bf] [ 0.000000] RAMDISK: [mem 0xbcc54000-0xbffbffff] [ 0.000000] ACPI: Early table checksum verification disabled [ 0.000000] ACPI: RSDP 0x00000000000F52D0 000014 (v00 BOCHS ) [ 0.000000] ACPI: RSDT 0x00000000BFFE247C 000034 (v01 BOCHS BXPC 00000001 BXPC 00000001) [ 0.000000] ACPI: FACP 0x00000000BFFE2318 000074 (v01 BOCHS BXPC 00000001 BXPC 00000001) [ 0.000000] ACPI: DSDT 0x00000000BFFE0040 0022D8 (v01 BOCHS BXPC 00000001 BXPC 00000001) [ 0.000000] ACPI: FACS 0x00000000BFFE0000 000040 [ 0.000000] ACPI: APIC 0x00000000BFFE238C 000090 (v03 BOCHS BXPC 00000001 BXPC 00000001) [ 0.000000] ACPI: HPET 0x00000000BFFE241C 000038 (v01 BOCHS BXPC 00000001 BXPC 00000001) [ 0.000000] ACPI: WAET 0x00000000BFFE2454 000028 (v01 BOCHS BXPC 00000001 BXPC 00000001) [ 0.000000] ACPI: Reserving FACP table memory at [mem 0xbffe2318-0xbffe238b] [ 0.000000] ACPI: Reserving DSDT table memory at [mem 0xbffe0040-0xbffe2317] [ 0.000000] ACPI: Reserving FACS table memory at [mem 0xbffe0000-0xbffe003f] [ 0.000000] ACPI: Reserving APIC table memory at [mem 0xbffe238c-0xbffe241b] [ 0.000000] ACPI: Reserving HPET table memory at [mem 0xbffe241c-0xbffe2453] [ 0.000000] ACPI: Reserving WAET table memory at [mem 0xbffe2454-0xbffe247b] [ 0.000000] No NUMA configuration found [ 0.000000] Faking a node at [mem 0x0000000000000000-0x0000000146dfffff] [ 0.000000] NODE_DATA(0) allocated [mem 0x1465a3000-0x1465cdfff] [ 0.000000] Reserving 256MB of memory at 2752MB for crashkernel (System RAM: 4205MB) [ 0.000000] Zone ranges: [ 0.000000] DMA [mem 0x0000000000001000-0x0000000000ffffff] [ 0.000000] DMA32 [mem 0x0000000001000000-0x00000000ffffffff] [ 0.000000] Normal [mem 0x0000000100000000-0x0000000146dfffff] [ 0.000000] Device empty [ 0.000000] Movable zone start for each node [ 0.000000] Early memory node ranges [ 0.000000] node 0: [mem 0x0000000000001000-0x000000000009efff] [ 0.000000] node 0: [mem 0x0000000000100000-0x00000000bffcdfff] [ 0.000000] node 0: [mem 0x0000000100000000-0x0000000146dfffff] [ 0.000000] Zeroed struct page in unavailable ranges: 4756 pages [ 0.000000] Initmem setup node 0 [mem 0x0000000000001000-0x0000000146dfffff] [ 0.000000] ACPI: PM-Timer IO Port: 0x608 [ 0.000000] ACPI: LAPIC_NMI (acpi_id[0xff] dfl dfl lint[0x1]) [ 0.000000] IOAPIC[0]: apic_id 0, version 17, address 0xfec00000, GSI 0-23 [ 0.000000] ACPI: INT_SRC_OVR (bus 0 bus_irq 0 global_irq 2 dfl dfl) [ 0.000000] ACPI: INT_SRC_OVR (bus 0 bus_irq 5 global_irq 5 high level) [ 0.000000] ACPI: INT_SRC_OVR (bus 0 bus_irq 9 global_irq 9 high level) [ 0.000000] ACPI: INT_SRC_OVR (bus 0 bus_irq 10 global_irq 10 high level) [ 0.000000] ACPI: INT_SRC_OVR (bus 0 bus_irq 11 global_irq 11 high level) [ 0.000000] Using ACPI (MADT) for SMP configuration information [ 0.000000] ACPI: HPET id: 0x8086a201 base: 0xfed00000 [ 0.000000] TSC deadline timer available [ 0.000000] smpboot: Allowing 4 CPUs, 0 hotplug CPUs [ 0.000000] kvm-guest: KVM setup pv remote TLB flush [ 0.000000] kvm-guest: setup PV sched yield [ 0.000000] PM: Registered nosave memory: [mem 0x00000000-0x00000fff] [ 0.000000] PM: Registered nosave memory: [mem 0x0009f000-0x0009ffff] [ 0.000000] PM: Registered nosave memory: [mem 0x000a0000-0x000effff] [ 0.000000] PM: Registered nosave memory: [mem 0x000f0000-0x000fffff] [ 0.000000] PM: Registered nosave memory: [mem 0xbffce000-0xbfffffff] [ 0.000000] PM: Registered nosave memory: [mem 0xc0000000-0xfeffbfff] [ 0.000000] PM: Registered nosave memory: [mem 0xfeffc000-0xfeffffff] [ 0.000000] PM: Registered nosave memory: [mem 0xff000000-0xfffbffff] [ 0.000000] PM: Registered nosave memory: [mem 0xfffc0000-0xffffffff] [ 0.000000] [mem 0xc0000000-0xfeffbfff] available for PCI devices [ 0.000000] Booting paravirtualized kernel on KVM [ 0.000000] clocksource: refined-jiffies: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 1910969940391419 ns [ 0.000000] setup_percpu: NR_CPUS:8192 nr_cpumask_bits:4 nr_cpu_ids:4 nr_node_ids:1 [ 0.000000] percpu: Embedded 63 pages/cpu s221184 r8192 d28672 u524288 [ 0.000000] kvm-guest: PV spinlocks enabled [ 0.000000] PV qspinlock hash table entries: 256 (order: 0, 4096 bytes, linear) [ 0.000000] Built 1 zonelists, mobility grouping on. Total pages: 1059606 [ 0.000000] Policy zone: Normal [ 0.000000] Kernel command line: rd.shell root=nbd:192.168.200.253:rocky8.10:ext4:ro:-p,-b4096 ro crashkernel=256M panic=1 nomodeset ipmtu=9000 ip=dhcp rd.neednet=1 init_on_free=off mitigations=off console=ttyS1,115200 audit=0 [ 0.000000] Specific versions of hardware are certified with Red Hat Enterprise Linux 8. Please see the list of hardware certified with Red Hat Enterprise Linux 8 at https://catalog.redhat.com. [ 0.000000] audit: disabled (until reboot) [ 0.000000] software IO TLB: area num 4. [ 0.000000] Memory: 2829652K/4306352K available (18435K kernel code, 11221K rwdata, 7248K rodata, 2908K init, 18040K bss, 524580K reserved, 0K cma-reserved) [ 0.000000] SLUB: HWalign=64, Order=0-3, MinObjects=0, CPUs=4, Nodes=1 [ 0.000000] kmemleak: Kernel memory leak detector disabled [ 0.000000] ftrace: allocating 41240 entries in 162 pages [ 0.000000] ftrace: allocated 162 pages with 3 groups [ 0.000000] rcu: Hierarchical RCU implementation. [ 0.000000] rcu: RCU event tracing is enabled. [ 0.000000] rcu: RCU restricting CPUs from NR_CPUS=8192 to nr_cpu_ids=4. [ 0.000000] rcu: RCU callback double-/use-after-free debug enabled. [ 0.000000] Rude variant of Tasks RCU enabled. [ 0.000000] Tracing variant of Tasks RCU enabled. [ 0.000000] rcu: RCU calculated value of scheduler-enlistment delay is 100 jiffies. [ 0.000000] rcu: Adjusting geometry for rcu_fanout_leaf=16, nr_cpu_ids=4 [ 0.000000] NR_IRQS: 524544, nr_irqs: 456, preallocated irqs: 16 [ 0.000000] random: get_random_bytes called from start_kernel+0x622/0x9a8 with crng_init=0 [ 0.001000] Console: colour *CGA 80x25 [ 0.001000] printk: console [ttyS1] enabled [ 0.001000] ACPI: Core revision 20220331 [ 0.001000] clocksource: hpet: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 19112604467 ns [ 0.001012] APIC: Switch to symmetric I/O mode setup [ 0.002372] x2apic enabled [ 0.003012] Switched APIC routing to physical x2apic. [ 0.004021] kvm-guest: setup PV IPIs [ 0.007706] ..TIMER: vector=0x30 apic1=0 pin1=2 apic2=-1 pin2=-1 [ 0.008000] clocksource: tsc-early: mask: 0xffffffffffffffff max_cycles: 0x229835b7123, max_idle_ns: 440795242976 ns [ 0.008033] Calibrating delay loop (skipped) preset value.. 4799.99 BogoMIPS (lpj=2399998) [ 0.009021] pid_max: default: 32768 minimum: 301 [ 0.010168] LSM: Security Framework initializing [ 0.011077] Yama: becoming mindful. [ 0.012059] SELinux: Initializing. [ 0.013108] *** VALIDATE selinux *** [ 0.022645] Dentry cache hash table entries: 1048576 (order: 11, 8388608 bytes, vmalloc) [ 0.027698] Inode-cache hash table entries: 524288 (order: 10, 4194304 bytes, vmalloc) [ 0.029125] Mount-cache hash table entries: 16384 (order: 5, 131072 bytes, vmalloc) [ 0.030132] Mountpoint-cache hash table entries: 16384 (order: 5, 131072 bytes, vmalloc) [ 0.031142] *** VALIDATE tmpfs *** [ 0.033395] *** VALIDATE proc *** [ 0.035115] *** VALIDATE cgroup *** [ 0.036018] *** VALIDATE cgroup2 *** [ 0.037311] x86/cpu: User Mode Instruction Prevention (UMIP) activated [ 0.038183] Last level iTLB entries: 4KB 0, 2MB 0, 4MB 0 [ 0.039014] Last level dTLB entries: 4KB 0, 2MB 0, 4MB 0, 1GB 0 [ 0.040043] Spectre V2 : User space: Vulnerable [ 0.041018] Speculative Store Bypass: Vulnerable [ 0.044316] debug: unmapping init [mem 0xffffffffa4e59000-0xffffffffa4e60fff] [ 0.046952] smpboot: CPU0: Intel(R) Xeon(R) CPU E5-2695 v2 @ 2.40GHz (family: 0x6, model: 0x3e, stepping: 0x4) [ 0.047769] Performance Events: IvyBridge events, full-width counters, Intel PMU driver. [ 0.048030] ... version: 2 [ 0.049021] ... bit width: 48 [ 0.050018] ... generic registers: 4 [ 0.051020] ... value mask: 0000ffffffffffff [ 0.052022] ... max period: 00007fffffffffff [ 0.053023] ... fixed-purpose events: 3 [ 0.054019] ... event mask: 000000070000000f [ 0.055384] rcu: Hierarchical SRCU implementation. [ 0.057767] smp: Bringing up secondary CPUs ... [ 0.058716] x86: Booting SMP configuration: [ 0.059041] .... node #0, CPUs: #1 #2 #3 [ 0.063159] smp: Brought up 1 node, 4 CPUs [ 0.065032] smpboot: Max logical packages: 1 [ 0.066022] smpboot: Total of 4 processors activated (19199.98 BogoMIPS) [ 0.222453] node 0 deferred pages initialised in 152ms [ 0.226166] devtmpfs: initialized [ 0.227289] x86/mm: Memory block size: 128MB [ 0.231025] gcov: version magic: 0x41383552 [ 0.233299] clocksource: jiffies: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 1911260446275000 ns [ 0.237121] futex hash table entries: 1024 (order: 4, 65536 bytes, vmalloc) [ 0.239321] pinctrl core: initialized pinctrl subsystem [ 0.241192] [ 0.241651] ************************************************************* [ 0.244015] ** NOTICE NOTICE NOTICE NOTICE NOTICE NOTICE NOTICE ** [ 0.246011] ** ** [ 0.248014] ** IOMMU DebugFS SUPPORT HAS BEEN ENABLED IN THIS KERNEL ** [ 0.250014] ** ** [ 0.252015] ** This means that this kernel is built to expose internal ** [ 0.254017] ** IOMMU data structures, which may compromise security on ** [ 0.256016] ** your system. ** [ 0.257017] ** ** [ 0.259015] ** If you see this message and you are not debugging the ** [ 0.261015] ** kernel, report this immediately to your vendor! ** [ 0.263013] ** ** [ 0.264011] ** NOTICE NOTICE NOTICE NOTICE NOTICE NOTICE NOTICE ** [ 0.267014] ************************************************************* [ 0.269700] NET: Registered protocol family 16 [ 0.271393] DMA: preallocated 512 KiB GFP_KERNEL pool for atomic allocations [ 0.274066] DMA: preallocated 512 KiB GFP_KERNEL|GFP_DMA pool for atomic allocations [ 0.276061] DMA: preallocated 512 KiB GFP_KERNEL|GFP_DMA32 pool for atomic allocations [ 0.280052] cpuidle: using governor menu [ 0.281834] acpiphp: ACPI Hot Plug PCI Controller Driver version: 0.5 [ 0.283529] PCI: Using configuration type 1 for base access [ 0.285133] core: PMU erratum BJ122, BV98, HSD29 worked around, HT is on [ 0.295110] HugeTLB registered 1.00 GiB page size, pre-allocated 0 pages [ 0.298063] HugeTLB registered 2.00 MiB page size, pre-allocated 0 pages [ 0.302060] cryptd: max_cpu_qlen set to 1000 [ 0.304196] ACPI: Added _OSI(Module Device) [ 0.306018] ACPI: Added _OSI(Processor Device) [ 0.308054] ACPI: Added _OSI(3.0 _SCP Extensions) [ 0.310022] ACPI: Added _OSI(Processor Aggregator Device) [ 0.317342] ACPI: 1 ACPI AML tables successfully acquired and loaded [ 0.325409] ACPI: Interpreter enabled [ 0.327085] ACPI: PM: (supports S0 S3 S4 S5) [ 0.329016] ACPI: Using IOAPIC for interrupt routing [ 0.331156] PCI: Using host bridge windows from ACPI; if necessary, use "pci=nocrs" and report a bug [ 0.334435] ACPI: Enabled 2 GPEs in block 00 to 0F [ 0.346700] ACPI: PCI Root Bridge [PCI0] (domain 0000 [bus 00-ff]) [ 0.349061] acpi PNP0A03:00: _OSC: OS supports [ASPM ClockPM Segments MSI HPX-Type3] [ 0.353031] acpi PNP0A03:00: _OSC: not requesting OS control; OS requires [ExtendedConfig ASPM ClockPM MSI] [ 0.356180] acpi PNP0A03:00: fail to add MMCONFIG information, can't access extended PCI configuration space under this bridge. [ 0.361472] acpiphp: Slot [2] registered [ 0.363182] acpiphp: Slot [5] registered [ 0.365281] acpiphp: Slot [6] registered [ 0.366193] acpiphp: Slot [7] registered [ 0.368228] acpiphp: Slot [8] registered [ 0.369117] acpiphp: Slot [9] registered [ 0.371176] acpiphp: Slot [10] registered [ 0.372134] acpiphp: Slot [3] registered [ 0.373112] acpiphp: Slot [4] registered [ 0.374146] acpiphp: Slot [11] registered [ 0.376124] acpiphp: Slot [12] registered [ 0.377097] acpiphp: Slot [13] registered [ 0.379148] acpiphp: Slot [14] registered [ 0.381133] acpiphp: Slot [15] registered [ 0.383151] acpiphp: Slot [16] registered [ 0.385374] acpiphp: Slot [17] registered [ 0.387146] acpiphp: Slot [18] registered [ 0.389165] acpiphp: Slot [19] registered [ 0.391142] acpiphp: Slot [20] registered [ 0.392118] acpiphp: Slot [21] registered [ 0.394105] acpiphp: Slot [22] registered [ 0.395118] acpiphp: Slot [23] registered [ 0.397158] acpiphp: Slot [24] registered [ 0.398141] acpiphp: Slot [25] registered [ 0.400146] acpiphp: Slot [26] registered [ 0.401120] acpiphp: Slot [27] registered [ 0.403123] acpiphp: Slot [28] registered [ 0.404116] acpiphp: Slot [29] registered [ 0.405135] acpiphp: Slot [30] registered [ 0.407123] acpiphp: Slot [31] registered [ 0.408050] PCI host bridge to bus 0000:00 [ 0.410026] pci_bus 0000:00: root bus resource [io 0x0000-0x0cf7 window] [ 0.412039] pci_bus 0000:00: root bus resource [io 0x0d00-0xffff window] [ 0.415036] pci_bus 0000:00: root bus resource [mem 0x000a0000-0x000bffff window] [ 0.417026] pci_bus 0000:00: root bus resource [mem 0xc0000000-0xfebfffff window] [ 0.419030] pci_bus 0000:00: root bus resource [mem 0xe0000000000-0xe007fffffff window] [ 0.421032] pci_bus 0000:00: root bus resource [bus 00-ff] [ 0.423185] pci 0000:00:00.0: [8086:1237] type 00 class 0x060000 [ 0.425220] pci 0000:00:01.0: [8086:7000] type 00 class 0x060100 [ 0.429706] pci 0000:00:01.1: [8086:7010] type 00 class 0x010180 [ 0.438650] pci 0000:00:01.1: reg 0x20: [io 0xc320-0xc32f] [ 0.444065] pci 0000:00:01.1: legacy IDE quirk: reg 0x10: [io 0x01f0-0x01f7] [ 0.447022] pci 0000:00:01.1: legacy IDE quirk: reg 0x14: [io 0x03f6] [ 0.448018] pci 0000:00:01.1: legacy IDE quirk: reg 0x18: [io 0x0170-0x0177] [ 0.450020] pci 0000:00:01.1: legacy IDE quirk: reg 0x1c: [io 0x0376] [ 0.452593] pci 0000:00:01.3: [8086:7113] type 00 class 0x068000 [ 0.455741] pci 0000:00:01.3: quirk: [io 0x0600-0x063f] claimed by PIIX4 ACPI [ 0.457043] pci 0000:00:01.3: quirk: [io 0x0700-0x070f] claimed by PIIX4 SMB [ 0.459772] pci 0000:00:02.0: [1af4:1000] type 00 class 0x020000 [ 0.463880] pci 0000:00:02.0: reg 0x10: [io 0xc300-0xc31f] [ 0.475021] pci 0000:00:02.0: reg 0x20: [mem 0xe0000000000-0xe0000003fff 64bit pref] [ 0.481020] pci 0000:00:02.0: reg 0x30: [mem 0xfeb80000-0xfebbffff pref] [ 0.486274] pci 0000:00:05.0: [1af4:1001] type 00 class 0x010000 [ 0.496023] pci 0000:00:05.0: reg 0x10: [io 0xc000-0xc07f] [ 0.503021] pci 0000:00:05.0: reg 0x14: [mem 0xfebc0000-0xfebc0fff] [ 0.526029] pci 0000:00:05.0: reg 0x20: [mem 0xe0000004000-0xe0000007fff 64bit pref] [ 0.537958] pci 0000:00:06.0: [1af4:1001] type 00 class 0x010000 [ 0.548024] pci 0000:00:06.0: reg 0x10: [io 0xc080-0xc0ff] [ 0.557018] pci 0000:00:06.0: reg 0x14: [mem 0xfebc1000-0xfebc1fff] [ 0.579022] pci 0000:00:06.0: reg 0x20: [mem 0xe0000008000-0xe000000bfff 64bit pref] [ 0.590735] pci 0000:00:07.0: [1af4:1001] type 00 class 0x010000 [ 0.597023] pci 0000:00:07.0: reg 0x10: [io 0xc100-0xc17f] [ 0.606021] pci 0000:00:07.0: reg 0x14: [mem 0xfebc2000-0xfebc2fff] [ 0.623018] pci 0000:00:07.0: reg 0x20: [mem 0xe000000c000-0xe000000ffff 64bit pref] [ 0.636754] pci 0000:00:08.0: [1af4:1001] type 00 class 0x010000 [ 0.653020] pci 0000:00:08.0: reg 0x10: [io 0xc180-0xc1ff] [ 0.663020] pci 0000:00:08.0: reg 0x14: [mem 0xfebc3000-0xfebc3fff] [ 0.696020] pci 0000:00:08.0: reg 0x20: [mem 0xe0000010000-0xe0000013fff 64bit pref] [ 0.714347] pci 0000:00:09.0: [1af4:1001] type 00 class 0x010000 [ 0.721020] pci 0000:00:09.0: reg 0x10: [io 0xc200-0xc27f] [ 0.730022] pci 0000:00:09.0: reg 0x14: [mem 0xfebc4000-0xfebc4fff] [ 0.748024] pci 0000:00:09.0: reg 0x20: [mem 0xe0000014000-0xe0000017fff 64bit pref] [ 0.762069] pci 0000:00:0a.0: [1af4:1001] type 00 class 0x010000 [ 0.771021] pci 0000:00:0a.0: reg 0x10: [io 0xc280-0xc2ff] [ 0.780023] pci 0000:00:0a.0: reg 0x14: [mem 0xfebc5000-0xfebc5fff] [ 0.803023] pci 0000:00:0a.0: reg 0x20: [mem 0xe0000018000-0xe000001bfff 64bit pref] [ 0.819807] ACPI: PCI: Interrupt link LNKA configured for IRQ 10 [ 0.822421] ACPI: PCI: Interrupt link LNKB configured for IRQ 10 [ 0.824418] ACPI: PCI: Interrupt link LNKC configured for IRQ 11 [ 0.827419] ACPI: PCI: Interrupt link LNKD configured for IRQ 11 [ 0.829257] ACPI: PCI: Interrupt link LNKS configured for IRQ 9 [ 0.834156] iommu: Default domain type: Passthrough [ 0.835000] SCSI subsystem initialized [ 0.836536] ACPI: bus type USB registered [ 0.838179] usbcore: registered new interface driver usbfs [ 0.841221] usbcore: registered new interface driver hub [ 0.843136] usbcore: registered new device driver usb [ 0.845236] pps_core: LinuxPPS API ver. 1 registered [ 0.848022] pps_core: Software ver. 5.3.6 - Copyright 2005-2007 Rodolfo Giometti [ 0.851090] PTP clock support registered [ 0.854087] EDAC MC: Ver: 3.0.0 [ 0.857050] PCI: Using ACPI for IRQ routing [ 0.859152] NetLabel: Initializing [ 0.860017] NetLabel: domain hash size = 128 [ 0.861014] NetLabel: protocols = UNLABELED CIPSOv4 CALIPSO [ 0.862197] NetLabel: unlabeled traffic allowed by default [ 0.864424] vgaarb: loaded [ 0.866307] hpet0: at MMIO 0xfed00000, IRQs 2, 8, 0 [ 0.868032] hpet0: 3 comparators, 64-bit 100.000000 MHz counter [ 0.873005] clocksource: Switched to clocksource kvm-clock [ 0.990672] VFS: Disk quotas dquot_6.6.0 [ 0.992330] VFS: Dquot-cache hash table entries: 512 (order 0, 4096 bytes) [ 0.994866] *** VALIDATE ramfs *** [ 0.996353] *** VALIDATE hugetlbfs *** [ 0.997897] pnp: PnP ACPI init [ 1.000785] pnp: PnP ACPI: found 6 devices [ 1.030436] clocksource: acpi_pm: mask: 0xffffff max_cycles: 0xffffff, max_idle_ns: 2085701024 ns [ 1.036171] pci_bus 0000:00: resource 4 [io 0x0000-0x0cf7 window] [ 1.038836] pci_bus 0000:00: resource 5 [io 0x0d00-0xffff window] [ 1.042160] pci_bus 0000:00: resource 6 [mem 0x000a0000-0x000bffff window] [ 1.047385] pci_bus 0000:00: resource 7 [mem 0xc0000000-0xfebfffff window] [ 1.049735] pci_bus 0000:00: resource 8 [mem 0xe0000000000-0xe007fffffff window] [ 1.052438] NET: Registered protocol family 2 [ 1.054930] IP idents hash table entries: 131072 (order: 8, 1048576 bytes, vmalloc) [ 1.059614] tcp_listen_portaddr_hash hash table entries: 4096 (order: 5, 163840 bytes, vmalloc) [ 1.063476] TCP established hash table entries: 65536 (order: 7, 524288 bytes, vmalloc) [ 1.069094] TCP bind hash table entries: 65536 (order: 9, 2097152 bytes, vmalloc) [ 1.072643] TCP: Hash tables configured (established 65536 bind 65536) [ 1.075893] MPTCP token hash table entries: 8192 (order: 6, 393216 bytes, vmalloc) [ 1.079368] UDP hash table entries: 4096 (order: 6, 393216 bytes, vmalloc) [ 1.082506] UDP-Lite hash table entries: 4096 (order: 6, 393216 bytes, vmalloc) [ 1.085191] NET: Registered protocol family 1 [ 1.088210] RPC: Registered named UNIX socket transport module. [ 1.090615] RPC: Registered udp transport module. [ 1.092522] RPC: Registered tcp transport module. [ 1.094350] RPC: Registered tcp NFSv4.1 backchannel transport module. [ 1.096956] NET: Registered protocol family 44 [ 1.098676] pci 0000:00:00.0: Limiting direct PCI/PCI transfers [ 1.100971] pci 0000:00:01.0: PIIX3: Enabling Passive Release [ 1.103160] pci 0000:00:01.0: Activating ISA DMA hang workarounds [ 1.105530] PCI: CLS 0 bytes, default 64 [ 1.107996] Unpacking initramfs... [ 2.556969] debug: unmapping init [mem 0xffff968cbcc54000-0xffff968cbffbffff] [ 2.563228] PCI-DMA: Using software bounce buffering for IO (SWIOTLB) [ 2.565345] software IO TLB: mapped [mem 0x00000000a8000000-0x00000000ac000000] (64MB) [ 2.569555] clocksource: tsc: mask: 0xffffffffffffffff max_cycles: 0x229835b7123, max_idle_ns: 440795242976 ns [ 3.061510] Initialise system trusted keyrings [ 3.062815] Key type blacklist registered [ 3.064474] workingset: timestamp_bits=36 max_order=20 bucket_order=0 [ 3.075312] zbud: loaded [ 3.078827] *** VALIDATE nfs *** [ 3.080442] *** VALIDATE nfs4 *** [ 3.082287] pstore: using deflate compression [ 3.085877] Platform Keyring initialized [ 3.192512] NET: Registered protocol family 38 [ 3.194600] Key type asymmetric registered [ 3.196201] Asymmetric key parser 'x509' registered [ 3.198429] Block layer SCSI generic (bsg) driver version 0.4 loaded (major 247) [ 3.202239] io scheduler mq-deadline registered [ 3.204198] io scheduler kyber registered [ 3.208241] io scheduler bfq registered [ 3.209842] atomic64_test: passed for x86-64 platform with CX8 and with SSE [ 3.212404] shpchp: Standard Hot Plug PCI Controller Driver version: 0.4 [ 3.215418] input: Power Button as /devices/LNXSYSTM:00/LNXPWRBN:00/input/input0 [ 3.218186] ACPI: Power Button [PWRF] [ 3.224776] ACPI: \_SB_.LNKB: Enabled at IRQ 10 [ 3.232343] ACPI: \_SB_.LNKA: Enabled at IRQ 11 [ 3.255956] ACPI: \_SB_.LNKC: Enabled at IRQ 11 [ 3.265474] ACPI: \_SB_.LNKD: Enabled at IRQ 10 [ 3.290770] Serial: 8250/16550 driver, 4 ports, IRQ sharing enabled [ 3.320369] 00:03: ttyS1 at I/O 0x2f8 (irq = 3, base_baud = 115200) is a 16550A [ 3.357565] 00:04: ttyS0 at I/O 0x3f8 (irq = 4, base_baud = 115200) is a 16550A [ 3.363130] Non-volatile memory driver v1.3 [ 3.364919] Linux agpgart interface v0.103 [ 3.404959] virtio_blk virtio1: [vda] 146048 512-byte logical blocks (74.8 MB/71.3 MiB) [ 3.409389] vda: detected capacity change from 0 to 74776576 [ 3.436452] virtio_blk virtio2: [vdb] 2097152 512-byte logical blocks (1.07 GB/1.00 GiB) [ 3.439664] vdb: detected capacity change from 0 to 1073741824 [ 3.459077] virtio_blk virtio3: [vdc] 5120000 512-byte logical blocks (2.62 GB/2.44 GiB) [ 3.464430] vdc: detected capacity change from 0 to 2621440000 [ 3.481596] virtio_blk virtio4: [vdd] 5120000 512-byte logical blocks (2.62 GB/2.44 GiB) [ 3.484789] vdd: detected capacity change from 0 to 2621440000 [ 3.510695] virtio_blk virtio5: [vde] 8388608 512-byte logical blocks (4.29 GB/4.00 GiB) [ 3.513805] vde: detected capacity change from 0 to 4294967296 [ 3.529779] virtio_blk virtio6: [vdf] 8388608 512-byte logical blocks (4.29 GB/4.00 GiB) [ 3.533194] vdf: detected capacity change from 0 to 4294967296 [ 3.540086] libphy: Fixed MDIO Bus: probed [ 3.561153] usbcore: registered new interface driver usbserial_generic [ 3.563120] usbserial: USB Serial support registered for generic [ 3.565805] i8042: PNP: PS/2 Controller [PNP0303:KBD,PNP0f13:MOU] at 0x60,0x64 irq 1,12 [ 3.571474] serio: i8042 KBD port at 0x60,0x64 irq 1 [ 3.573593] serio: i8042 AUX port at 0x60,0x64 irq 12 [ 3.576039] mousedev: PS/2 mouse device common for all mice [ 3.578511] rtc_cmos 00:05: RTC can wake from S4 [ 3.581316] rtc_cmos 00:05: registered as rtc0 [ 3.582982] rtc_cmos 00:05: alarms up to one day, y3k, 242 bytes nvram, hpet irqs [ 3.585917] intel_pstate: CPU model not supported [ 3.589506] hid: raw HID events driver (C) Jiri Kosina [ 3.591575] usbcore: registered new interface driver usbhid [ 3.592179] input: AT Translated Set 2 keyboard as /devices/platform/i8042/serio0/input/input1 [ 3.596132] usbhid: USB HID core driver [ 3.596345] drop_monitor: Initializing network drop monitor service [ 3.596513] Initializing XFRM netlink socket [ 3.596865] NET: Registered protocol family 10 [ 3.599180] Segment Routing with IPv6 [ 3.607539] NET: Registered protocol family 17 [ 3.610097] input: VirtualPS/2 VMware VMMouse as /devices/platform/i8042/serio1/input/input4 [ 3.611171] mpls_gso: MPLS GSO support [ 3.617946] input: VirtualPS/2 VMware VMMouse as /devices/platform/i8042/serio1/input/input3 [ 3.622268] RAS: Correctable Errors collector initialized. [ 3.624291] AVX version of gcm_enc/dec engaged. [ 3.625662] AES CTR mode by8 optimization enabled [ 3.706563] sched_clock: Marking stable (3706461043, 0)->(4720713214, -1014252171) [ 3.710836] registered taskstats version 1 [ 3.713173] Loading compiled-in X.509 certificates [ 3.715411] zswap: loaded using pool lzo/zbud [ 3.738438] Key type big_key registered [ 3.752856] Key type encrypted registered [ 3.754083] ima: No TPM chip found, activating TPM-bypass! [ 3.755607] ima: Allocated hash algorithm: sha1 [ 3.757232] ima: No architecture policies found [ 3.759129] evm: Initialising EVM extended attributes: [ 3.761211] evm: security.selinux [ 3.762565] evm: security.ima [ 3.763656] evm: security.capability [ 3.764897] evm: HMAC attrs: 0x1 [ 3.767245] rtc_cmos 00:05: setting system clock to 2026-08-20 01:44:37 UTC (1787190277) [ 3.772898] debug: unmapping init [mem 0xffffffffa5e03000-0xffffffffa5ffffff] [ 3.775661] debug: unmapping init [mem 0xffffffffa4b82000-0xffffffffa4e58fff] [ 3.784073] Write protecting the kernel read-only data: 28672k [ 3.787486] debug: unmapping init [mem 0xffffffffa3203000-0xffffffffa33fffff] [ 3.790481] debug: unmapping init [mem 0xffffffffa3b14000-0xffffffffa3bfffff] [ 3.823148] systemd[1]: systemd 239 (239-82.el8_10.5) running in system mode. (+PAM +AUDIT +SELINUX +IMA -APPARMOR +SMACK +SYSVINIT +UTMP +LIBCRYPTSETUP +GCRYPT +GNUTLS +ACL +XZ +LZ4 +SECCOMP +BLKID +ELFUTILS +KMOD +IDN2 -IDN +PCRE2 default-hierarchy=legacy) [ 3.832694] systemd[1]: Detected virtualization kvm. [ 3.834178] systemd[1]: Detected architecture x86-64. [ 3.835652] systemd[1]: Running in initial RAM disk. Welcome to Rocky Linux 8.10 (Green Obsidian) dracut-049-233.git20240115.el8 (Initramfs)! [ 3.870371] systemd[1]: No hostname configured. [ 3.871866] systemd[1]: Set hostname to . [ 3.873659] random: systemd: uninitialized urandom read (16 bytes read) [ 3.875620] systemd[1]: Initializing machine ID from random generator. [ 3.998673] random: systemd: uninitialized urandom read (16 bytes read) [ 4.001981] systemd[1]: Listening on Journal Socket. [ OK ] Listening on Journal Socket. [ 4.006222] random: systemd: uninitialized urandom read (16 bytes read) [ 4.013214] systemd[1]: Started Memstrack Anylazing Service. [ OK ] Started Memstrack Anylazing Service. [ 4.023587] systemd[1]: Starting Setup Virtual Console... Starting Setup Virtual Console... [ OK ] Reached target Local File Systems. Starting Create Volatile Files and Directories... [ OK ] Reached target Swap. [ OK ] Listening on udev Control Socket. [ OK ] Listening on Journal Socket (/dev/log). Starting Journal Service... [ OK ] Reached target Timers. [ OK ] Reached target Slices. [ OK ] Listening on udev Kernel Socket. [ OK ] Reached target Sockets. [ OK ] Started Dispatch Password Requests to Console Directory Watch. [ OK ] Reached target Paths. [ OK ] Reached target Local Encrypted Volumes. [ OK ] Reached target Initrd Root Device. Starting Apply Kernel Variables... Starting Create list of required st…ce nodes for the current kernel... [ OK ] Started Setup Virtual Console. [ OK ] Started Create Volatile Files and Directories. [ OK ] Started Apply Kernel Variables. [ OK ] Started Create list of required sta…vice nodes for the current kernel. Starting Create Static Device Nodes in /dev... Starting dracut cmdline hook... [ OK ] Started Journal Service. [ OK ] Started Create Static Device Nodes in /dev. [ OK ] Started dracut cmdline hook. Starting dracut pre-udev hook... [ 4.535903] random: fast init done [ 4.583707] device-mapper: uevent: version 1.0.3 [ 4.585856] device-mapper: ioctl: 4.46.0-ioctl (2022-02-22) initialised: dm-devel@redhat.com [ OK ] Started dracut pre-udev hook. Starting udev Kernel Device Manager... [ OK ] Started udev Kernel Device Manager. Starting dracut pre-trigger hook... [ OK ] Started dracut pre-trigger hook. Starting udev Coldplug all Devices... Mounting Kernel Configuration File System... [ OK ] Mounted Kernel Configuration File System. [ OK ] Started udev Coldplug all Devices. Starting dracut initqueue hook... [ OK ] Reached target System Initialization. [ OK ] Reached target Basic System. [ OK ] Started Hardware RNG Entropy Gatherer Daemon. [ 5.303526] virtio_net virtio0 ens2: renamed from eth0 [ 5.334371] scsi host0: ata_piix [ 5.367703] scsi host1: ata_piix [ 5.368942] ata1: PATA max MWDMA2 cmd 0x1f0 ctl 0x3f6 bmdma 0xc320 irq 14 [ 5.370853] ata2: PATA max MWDMA2 cmd 0x170 ctl 0x376 bmdma 0xc328 irq 15 [ 9.653217] dracut-initqueue[597]: RTNETLINK answers: File exists [ 10.288348] random: crng init done [ 10.289669] random: 7 urandom warning(s) missed due to ratelimiting Starting nbd nbd0... [ OK ] Started nbd nbd0. [ OK ] Started dracut initqueue hook. Mounting /sysroot... [ OK ] Reached target Remote File Systems (Pre). [ OK ] Reached target Remote File Systems. [ 10.615498] EXT4-fs (nbd0): mounted filesystem with ordered data mode. Opts: (null) [ OK ] Mounted /sysroot. [ OK ] Reached target Initrd Root File System. Starting Reload Configuration from the Real Root... [ OK ] Started Reload Configuration from the Real Root. [ OK ] Reached target Initrd File Systems. [ OK ] Reached target Initrd Default Target. Starting dracut pre-pivot and cleanup hook... [ OK ] Started dracut pre-pivot and cleanup hook. Starting Cleaning Up and Shutting Down Daemons... [ OK ] Stopped dracut pre-pivot and cleanup hook. [ OK ] Stopped target Initrd Default Target. Stopping Hardware RNG Entropy Gatherer Daemon... [ OK ] Stopped target Timers. [ OK ] Stopped target Initrd Root Device. [ OK ] Stopped target Remote File Systems. [ OK ] Stopped target Remote File Systems (Pre). [ OK ] Stopped dracut initqueue hook. [ OK ] Stopped Hardware RNG Entropy Gatherer Daemon. [ OK ] Stopped target Basic System. [ OK ] Stopped target Sockets. [ OK ] Stopped target Slices. [ OK ] Stopped target Paths. [ OK ] Stopped target System Initialization. [ OK ] Stopped Create Volatile Files and Directories. [ OK ] Stopped target Swap. [ OK ] Stopped udev Coldplug all Devices. [ OK ] Stopped dracut pre-trigger hook. Stopping udev Kernel Device Manager... [ OK ] Stopped Apply Kernel Variables. [ OK ] Stopped target Local File Systems. [ OK ] Stopped target Local Encrypted Volumes. [ OK ] Stopped Dispatch Password Requests to Console Directory Watch. [ OK ] Started Cleaning Up and Shutting Down Daemons. [ OK ] Stopped udev Kernel Device Manager. [ OK ] Stopped dracut pre-udev hook. [ OK ] Stopped dracut cmdline hook. [ OK ] Stopped Create Static Device Nodes in /dev. [ OK ] Stopped Create list of required sta…vice nodes for the current kernel. [ OK ] Closed udev Kernel Socket. [ OK ] Closed udev Control Socket. Starting Cleanup udevd DB... [ OK ] Started Cleanup udevd DB. [ OK ] Reached target Switch Root. Starting Switch Root... [ 11.747460] printk: systemd: 25 output lines suppressed due to ratelimiting [ 12.013122] SELinux: Disabled at runtime. [ 12.073865] systemd[1]: systemd 239 (239-82.el8_10.5) running in system mode. (+PAM +AUDIT +SELINUX +IMA -APPARMOR +SMACK +SYSVINIT +UTMP +LIBCRYPTSETUP +GCRYPT +GNUTLS +ACL +XZ +LZ4 +SECCOMP +BLKID +ELFUTILS +KMOD +IDN2 -IDN +PCRE2 default-hierarchy=legacy) [ 12.082489] systemd[1]: Detected virtualization kvm. [ 12.084385] systemd[1]: Detected architecture x86-64. Welcome to Rocky Linux 8.10 (Green Obsidian)! [ 12.620661] systemd[1]: initrd-switch-root.service: Succeeded. [ 12.624839] systemd[1]: Stopped Switch Root. [ OK ] Stopped Switch Root. [ 12.634367] systemd[1]: systemd-journald.service: Service has no hold-off time (RestartSec=0), scheduling restart. [ 12.639455] systemd[1]: systemd-journald.service: Scheduled restart job, restart counter is at 1. [ 12.644380] systemd[1]: Stopped Journal Service. [ OK ] Stopped Journal Service. [ 12.653657] systemd[1]: Starting Journal Service... Starting Journal Service... [ 12.660284] systemd[1]: Created slice system-serial\x2dgetty.slice. [ OK ] Created slice system-serial\x2dgetty.slice. [ OK ] Started Dispatch Password Requests to Console Directory Watch. Mounting Huge Pages File System... [FAILED] Failed to set up automount Arbitrar…rmats File System Automount Point. See 'systemctl status proc-sys-fs-binfmt_misc.automount' for details. Starting Remount Root and Kernel File Systems... Starting Create list of required st…ce nodes for the current kernel... [ OK ] Started Forward Password Requests to Wall Directory Watch. [ OK ] Reached target Paths. [ OK ] Created slice system-getty.slice. Mounting POSIX Message Queue File System... Activating swap /dev/disk/by-label/SWAP... [ OK ] Listening on RPCbind Server Activation Socket. [ OK ] Reached target RPC Port Mapper. [ OK ] Created slice User and Session Slice. [ OK ] Reached target Slices. [ OK ] Reached target rpc_pipefs.target. [ OK ] Listening on udev Kernel Socket. [ OK ] Listening on udev Control Socket. Starting udev Coldplug all Devices... [ 12.801460] Adding 1048572k swap on /dev/vdb. Priority:-2 extents:1 across:1048572k FS [ OK ] Listening on Process Core Dump Socket. [ OK ] Stopped target Switch Root. [ OK ] Stopped target Initrd File Systems. Mounting Kernel Debug File System... Starting Apply Kernel Variables... [ OK ] Listening on initctl Compatibility Named Pipe. [ OK ] Reached target Local Encrypted Volumes. [ OK ] Stopped target Initrd Root File System. [ OK ] Created slice system-sshd\x2dkeygen.slice. [ OK ] Started Journal Service. [ OK ] Mounted Huge Pages File System. [FAILED] Failed to start Remount Root and Kernel File Systems. See 'systemctl status systemd-remount-fs.service' for details. [ OK ] Started Create list of required sta…vice nodes for the current kernel. [ OK ] Mounted POSIX Message Queue File System. [ OK ] Activated swap /dev/disk/by-label/SWAP. [ OK ] Mounted Kernel Debug File System. [ OK ] Started Apply Kernel Variables. [ OK ] Reached target Swap. Starting Configure read-only root support... Starting Create Static Device Nodes in /dev... Starting Flush Journal to Persistent Storage... [ OK ] Started Flush Journal to Persistent Storage. [ OK ] Started Create Static Device Nodes in /dev. Starting udev Kernel Device Manager... [ OK ] Reached target Local File Systems (Pre). Mounting /mnt... Mounting /home/green/git/lustre-release... [ OK ] Started udev Coldplug all Devices. [ OK ] Mounted /mnt. [ 13.178269] squashfs: version 4.0 (2009/01/31) Phillip Lougher [ OK ] Mounted /home/green/git/lustre-release. [ OK ] Started udev Kernel Device Manager. [ 13.459598] piix4_smbus 0000:00:01.3: SMBus Host Controller at 0x700, revision 0 [ 13.486232] input: PC Speaker as /devices/platform/pcspkr/input/input5 [ 13.648824] RAPL PMU: API unit is 2^-32 Joules, 0 fixed counters, 10737418240 ms ovfl timer [ 13.659539] EDAC sbridge: Ver: 1.1.2 [ 15.434414] Key type dns_resolver registered [ 15.775221] NFS: Registering the id_resolver key type [ 15.778216] Key type id_resolver registered [ 15.780776] Key type id_legacy registered [ OK ] Started Configure read-only root support. Starting Load/Save Random Seed... [ OK ] Reached target Local File Systems. Starting Create Volatile Files and Directories... Starting Rebuild Dynamic Linker Cache... Starting Mark the need to relabel after reboot... [ OK ] Started Load/Save Random Seed. [ OK ] Started Mark the need to relabel after reboot. [ OK ] Started Create Volatile Files and Directories. Starting RPC Bind... Starting Update UTMP about System Boot/Shutdown... [ OK ] Started Update UTMP about System Boot/Shutdown. [ OK ] Started RPC Bind. [ OK ] Started Rebuild Dynamic Linker Cache. Starting Update is Completed... [ OK ] Started Update is Completed. [ OK ] Reached target System Initialization. [ OK ] Started daily update of the root trust anchor for DNSSEC. [ OK ] Started dnf makecache --timer. [ OK ] Listening on D-Bus System Message Bus Socket. [ OK ] Reached target Sockets. [ OK ] Reached target Basic System. Starting Login Service... [ OK ] Started irqbalance daemon. [ OK ] Started Hardware RNG Entropy Gatherer Daemon. Starting Restore /run/initramfs on shutdown... [ OK ] Reached target sshd-keygen.target. [ OK ] Started D-Bus System Message Bus. [ OK ] Started Daily Cleanup of Temporary Directories. [ OK ] Reached target Timers. Starting Network Manager... [ OK ] Started Restore /run/initramfs on shutdown. [ OK ] Started Login Service. [ OK ] Started Network Manager. Starting Network Manager Wait Online... [ OK ] Reached target Network. Starting Dynamic System Tuning Daemon... Starting GSSAPI Proxy Daemon... Starting OpenSSH server daemon... Starting Hostname Service... [ OK ] Started GSSAPI Proxy Daemon. [ OK ] Started OpenSSH server daemon. [ OK ] Reached target NFS client services. [ OK ] Reached target Remote File Systems (Pre). [ OK ] Reached target Remote File Systems. Starting Permit User Sessions... [ OK ] Started Permit User Sessions. [ OK ] Started Serial Getty on ttyS0. [ OK ] Started Command Scheduler. [ OK ] Started Getty on tty1. [ OK ] Started Serial Getty on ttyS1. [ OK ] Reached target Login Prompts. [ OK ] Started Hostname Service. Starting Network Manager Script Dispatcher Service... [ OK ] Started Network Manager Script Dispatcher Service. [ OK ] Started Network Manager Wait Online. [ OK ] Reached target Network is Online. Starting Crash recovery kernel arming... Starting Notify NFS peers of a restart... Starting System Logging Service... [ OK ] Started Notify NFS peers of a restart. [ OK ] Started System Logging Service. Starting Authorization Manager... [ OK ] Started Authorization Manager. [ OK ] Started Dynamic System Tuning Daemon. [ OK ] Reached target Multi-User System. [ OK ] Reached target Graphical Interface. Starting Update UTMP about System Runlevel Changes... [ OK ] Started Update UTMP about System Runlevel Changes. Rocky Linux 8.10 (Green Obsidian) Kernel 4.18.0rh8.10-debug on an x86_64 oleg635-server login: [ 50.490410] libcfs: loading out-of-tree module taints kernel. [ 50.562222] Key type ._llcrypt registered [ 50.565586] Key type .llcrypt registered [ 50.652411] Lustre: DEBUG MARKER: oleg635-server.virtnet: executing set_hostid [ 69.974601] Lustre: DEBUG MARKER: oleg635-server.virtnet: executing load_modules_local [ 71.453352] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 71.483785] alg: No test for adler32 (adler32-zlib) [ 73.110692] Lustre: Lustre: Build Version: 2.17.57_44_gafaaa93 [ 74.065793] LNet: Added LNI 192.168.206.135@tcp [8/256/0/180] [ 74.361019] hrtimer: interrupt took 3610920 ns [ 75.983204] Key type lgssc registered [ 78.019334] Lustre: Echo OBD driver; http://www.lustre.org/ [ 95.125093] ZFS: Loaded module v2.3.2-1, ZFS pool version 5000, ZFS filesystem version 5 [ 133.311235] Lustre: DEBUG MARKER: oleg635-server.virtnet: executing load_modules_local [ 145.270811] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 145.323680] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 146.635348] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 146.672726] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 146.684680] Lustre: lustre-MDT0000: new disk, initializing [ 146.745335] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 146.764849] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 150.817311] Lustre: DEBUG MARKER: oleg635-server.virtnet: executing set_default_debug vfstrace rpctrace dlmtrace neterror ha config ioctl super lfsck all [ 164.232251] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 164.293892] Lustre: 6515:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 164.323810] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [ 164.329582] Lustre: Skipped 1 previous similar message [ 164.335269] Lustre: lustre-MDT0001: new disk, initializing [ 164.382960] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 164.420165] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [ 164.444575] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 168.372230] Lustre: DEBUG MARKER: oleg635-server.virtnet: executing set_default_debug vfstrace rpctrace dlmtrace neterror ha config ioctl super lfsck all [ 173.043463] Lustre: Modifying parameter general.debug_raw_pointers=Y in log params [ 181.352106] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 181.583823] Lustre: lustre-OST0000: new disk, initializing [ 181.588572] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 181.594556] Lustre: 8451:0:(osd_compat.c:1353:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 181.648058] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 186.498506] Lustre: DEBUG MARKER: oleg635-server.virtnet: executing set_default_debug vfstrace rpctrace dlmtrace neterror ha config ioctl super lfsck all [ 187.918855] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [ 187.930375] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 187.962628] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 198.756818] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 198.861468] Lustre: lustre-OST0001: new disk, initializing [ 198.864049] Lustre: srv-lustre-OST0001: No data found on store. Initialize space. [ 198.872488] Lustre: 9524:0:(osd_compat.c:1353:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 198.916055] Lustre: lustre-OST0001: Imperative Recovery not enabled, recovery window 60-180 [ 204.586989] Lustre: DEBUG MARKER: oleg635-server.virtnet: executing set_default_debug vfstrace rpctrace dlmtrace neterror ha config ioctl super lfsck all [ 207.366625] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x00000002c0000400-0x0000000300000400]:1:ost [ 207.373459] Lustre: cli-lustre-OST0001-super: Allocated super-sequence [0x00000002c0000400-0x0000000300000400]:1:ost] [ 207.420714] Lustre: lustre-OST0001-osc-MDT0000: update sequence from 0x100010000 to 0x2c0000401 [ 214.547718] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 221.795305] Lustre: Setting parameter general.lod.*.mdt_hash=crush in log params [ 227.051458] Lustre: DEBUG MARKER: oleg635-server.virtnet: executing check_logdir /tmp/testlogs/ [ 230.388506] Lustre: DEBUG MARKER: oleg635-server.virtnet: executing yml_node [ 233.627409] Lustre: DEBUG MARKER: Client: 2.17.57.44 [ 235.491347] Lustre: DEBUG MARKER: MDS: 2.17.57.44 [ 237.368176] Lustre: DEBUG MARKER: OSS: 2.17.57.44 [ 238.428575] Lustre: DEBUG MARKER: -----============= acceptance-small: conf-sanity ============----- Wed Aug 19 21:48:30 EDT 2026 [ 252.306277] Lustre: DEBUG MARKER: excepting tests: 102 106 115 32newtarball [ 254.004295] Lustre: DEBUG MARKER: skipping tests SLOW=no: 45 69 106 111 114 [ 271.843839] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 271.848310] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 271.864297] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 273.890613] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 273.892794] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 273.911397] Lustre: Skipped 1 previous similar message [ 273.924307] Lustre: Skipped 3 previous similar messages [ 275.426321] Lustre: server umount lustre-MDT0000 complete [ 278.812831] LustreError: 6508:0:(ldlm_lockd.c:2580:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1787190552 with bad export cookie 2193551412158363468 [ 278.814819] LustreError: MGC192.168.206.135@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 278.829739] LustreError: 6508:0:(ldlm_lockd.c:2580:ldlm_cancel_handler()) Skipped 1 previous similar message [ 279.007779] Lustre: lustre-MDT0001-lwp-OST0000: Connection to lustre-MDT0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 279.008333] Lustre: lustre-MDT0001: Not available for connect from 0@lo (stopping) [ 279.048312] Lustre: Skipped 2 previous similar messages [ 283.041966] Lustre: lustre-MDT0001: Not available for connect from 0@lo (stopping) [ 283.045744] Lustre: Skipped 2 previous similar messages [ 288.224410] LustreError: 6522:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 288.224508] Lustre: lustre-MDT0001: Not available for connect from 0@lo (stopping) [ 288.260391] LustreError: 6522:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 5 previous similar messages [ 288.281173] Lustre: Skipped 1 previous similar message [ 289.249271] LustreError: 6520:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 289.272280] LustreError: 6520:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 2 previous similar messages [ 293.344312] Lustre: lustre-MDT0001 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 293.527548] Lustre: server umount lustre-MDT0001 complete [ 297.373987] Lustre: server umount lustre-OST0000 complete [ 301.258494] Lustre: server umount lustre-OST0001 complete [ 307.723244] Lustre: DEBUG MARKER: oleg635-server.virtnet: executing set_hostid [ 316.094845] Lustre: DEBUG MARKER: oleg635-server.virtnet: executing load_modules_local [ 348.597249] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 348.805280] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 348.832274] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 348.838399] Lustre: lustre-MDT0000: new disk, initializing [ 348.904296] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 348.919841] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 352.893302] Lustre: DEBUG MARKER: oleg635-server.virtnet: executing set_default_debug -1 all [ 362.570728] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 362.707177] Lustre: 15074:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 362.771675] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [ 362.777214] Lustre: Skipped 1 previous similar message [ 362.781864] Lustre: lustre-MDT0001: new disk, initializing [ 362.882113] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 362.957620] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [ 362.973609] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 367.171659] Lustre: DEBUG MARKER: oleg635-server.virtnet: executing set_default_debug -1 all [ 376.751462] Lustre: DEBUG MARKER: oleg635-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 382.635648] Lustre: DEBUG MARKER: oleg635-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 388.507236] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 388.740783] Lustre: lustre-OST0000: new disk, initializing [ 388.744209] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 388.748945] Lustre: 17043:0:(osd_compat.c:1353:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 388.804898] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 390.143952] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [ 390.158523] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 390.260103] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 393.954144] Lustre: DEBUG MARKER: oleg635-server.virtnet: executing set_default_debug -1 all [ 401.944569] Lustre: DEBUG MARKER: oleg635-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 406.360229] Lustre: DEBUG MARKER: oleg635-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 [ 406.526189] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [ 411.080392] Lustre: DEBUG MARKER: oleg635-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid 50 [ 411.287941] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [ 415.728049] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 415.733159] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 415.745647] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 415.759686] Lustre: Skipped 1 previous similar message [ 415.764080] Lustre: Skipped 2 previous similar messages [ 426.975455] Lustre: lustre-OST0000 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 427.102846] Lustre: server umount lustre-OST0000 complete [ 431.075669] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 431.092971] Lustre: Skipped 1 previous similar message [ 436.196511] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 436.200123] Lustre: Skipped 8 previous similar messages [ 436.641630] Lustre: server umount lustre-MDT0000 complete [ 439.835642] LustreError: 15871:0:(ldlm_lockd.c:2580:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1787190713 with bad export cookie 2193551412158364994 [ 439.837460] LustreError: MGC192.168.206.135@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 439.845212] LustreError: 15871:0:(ldlm_lockd.c:2580:ldlm_cancel_handler()) Skipped 1 previous similar message [ 440.074952] Lustre: server umount lustre-MDT0001 complete [ 444.001332] Lustre: DEBUG MARKER: == conf-sanity test 76a: set permanent params with lctl across mounts ========================================================== 21:51:56 (1787190716) [ 449.360290] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 449.711012] LustreError: 19310:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 449.768441] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 452.924258] Lustre: DEBUG MARKER: oleg635-server.virtnet: executing set_default_debug -1 all [ 455.140087] LustreError: 19311:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 459.061402] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 462.479953] Lustre: DEBUG MARKER: oleg635-server.virtnet: executing set_default_debug -1 all [ 468.184639] Lustre: DEBUG MARKER: oleg635-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 473.255863] Lustre: DEBUG MARKER: oleg635-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 478.680131] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 484.170292] Lustre: DEBUG MARKER: oleg635-server.virtnet: executing set_default_debug -1 all [ 490.654493] Lustre: DEBUG MARKER: oleg635-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 498.536862] Lustre: Modifying parameter general.osc.*.max_dirty_mb=452 in log params [ 505.318537] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 505.322504] LustreError: Skipped 1 previous similar message [ 505.329811] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 505.344066] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 509.924852] Lustre: server umount lustre-MDT0000 complete [ 510.947095] LustreError: 20037:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 510.994343] LustreError: 20037:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 2 previous similar messages [ 514.124413] LustreError: 19292:0:(ldlm_lockd.c:2580:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1787190787 with bad export cookie 2193551412158365883 [ 514.129658] LustreError: MGC192.168.206.135@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 514.132364] LustreError: 19292:0:(ldlm_lockd.c:2580:ldlm_cancel_handler()) Skipped 1 previous similar message [ 514.303278] Lustre: server umount lustre-MDT0001 complete [ 527.600772] Lustre: server umount lustre-OST0000 complete [ 545.537968] Lustre: DEBUG MARKER: oleg635-server.virtnet: executing load_modules_local [ 553.681275] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 553.918313] LustreError: 23691:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 553.971507] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 553.978417] Lustre: Skipped 2 previous similar messages [ 557.582930] Lustre: DEBUG MARKER: oleg635-server.virtnet: executing set_default_debug -1 all [ 564.269460] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 567.759868] Lustre: DEBUG MARKER: oleg635-server.virtnet: executing set_default_debug -1 all [ 569.966204] Lustre: Setting parameter general.debug_raw_pointers=Y in log params [ 574.810737] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 574.952305] LustreError: 25180:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-OST0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 574.970294] LustreError: 25180:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 2 previous similar messages [ 579.629523] Lustre: DEBUG MARKER: oleg635-server.virtnet: executing set_default_debug -1 all [ 585.675732] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 585.846667] Lustre: lustre-OST0001: new disk, initializing [ 585.857692] Lustre: srv-lustre-OST0001: No data found on store. Initialize space. [ 585.868649] Lustre: 26010:0:(osd_compat.c:1353:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 587.272363] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x00000002c0000400-0x0000000300000400]:1:ost [ 587.277589] Lustre: cli-lustre-OST0001-super: Allocated super-sequence [0x00000002c0000400-0x0000000300000400]:1:ost] [ 587.346495] Lustre: lustre-OST0001-osc-MDT0000: update sequence from 0x100010000 to 0x2c0000401 [ 590.545659] Lustre: DEBUG MARKER: oleg635-server.virtnet: executing set_default_debug -1 all [ 597.148242] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 599.174951] Lustre: Setting parameter general.lod.*.mdt_hash=crush in log params [ 607.714051] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 607.720838] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 607.723114] Lustre: Skipped 5 previous similar messages [ 607.730923] Lustre: Skipped 5 previous similar messages [ 612.738112] Lustre: server umount lustre-MDT0000 complete [ 612.837792] LustreError: 25918:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 612.856325] LustreError: 25918:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 3 previous similar messages [ 615.454328] LustreError: 23672:0:(ldlm_lockd.c:2580:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1787190889 with bad export cookie 2193551412158366723 [ 615.456594] LustreError: MGC192.168.206.135@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 615.465599] LustreError: 23672:0:(ldlm_lockd.c:2580:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 615.592145] Lustre: server umount lustre-MDT0001 complete [ 628.316511] Lustre: server umount lustre-OST0000 complete [ 641.754951] Lustre: server umount lustre-OST0001 complete [ 654.218851] Lustre: DEBUG MARKER: oleg635-server.virtnet: executing load_modules_local [ 662.838761] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 667.135836] Lustre: DEBUG MARKER: oleg635-server.virtnet: executing set_default_debug -1 all [ 674.462104] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 678.919407] Lustre: DEBUG MARKER: oleg635-server.virtnet: executing set_default_debug -1 all [ 681.680748] Lustre: 30175:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify general/debug_raw_pointers=Y (mode = 0) failed: rc = -17 [ 687.073214] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 687.391581] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 687.397450] Lustre: Skipped 5 previous similar messages [ 688.431298] LustreError: 30529:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 688.448292] LustreError: 30529:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 3 previous similar messages [ 692.213729] Lustre: DEBUG MARKER: oleg635-server.virtnet: executing set_default_debug -1 all [ 699.207419] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 704.985740] Lustre: DEBUG MARKER: oleg635-server.virtnet: executing set_default_debug -1 all [ 711.929426] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 714.553350] Lustre: 32056:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify general/lod.*.mdt_hash=crush (mode = 0) failed: rc = -17 [ 720.872237] Lustre: lustre-MDT0000-lwp-OST0000: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 720.892504] Lustre: Skipped 2 previous similar messages [ 720.896620] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 725.304701] Lustre: server umount lustre-MDT0000 complete [ 728.128143] LustreError: 29007:0:(ldlm_lockd.c:2580:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1787191001 with bad export cookie 2193551412158368025 [ 728.133121] LustreError: MGC192.168.206.135@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 728.139654] LustreError: 29007:0:(ldlm_lockd.c:2580:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 728.269515] Lustre: server umount lustre-MDT0001 complete [ 740.379707] Lustre: server umount lustre-OST0000 complete [ 753.941852] Lustre: server umount lustre-OST0001 complete [ 760.403060] Lustre: DEBUG MARKER: == conf-sanity test 76b: verify params log setup correctly ========================================================== 21:57:12 (1787191032) [ 779.760736] Lustre: DEBUG MARKER: oleg635-server.virtnet: executing load_modules_local [ 787.192877] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 791.342966] Lustre: DEBUG MARKER: oleg635-server.virtnet: executing set_default_debug -1 all [ 798.446132] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 803.016553] Lustre: DEBUG MARKER: oleg635-server.virtnet: executing set_default_debug -1 all [ 805.954565] Lustre: 36001:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify general/debug_raw_pointers=Y (mode = 0) failed: rc = -17 [ 812.532035] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 817.479429] Lustre: DEBUG MARKER: oleg635-server.virtnet: executing set_default_debug -1 all [ 819.170850] LustreError: 36355:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 819.187544] LustreError: 36355:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 10 previous similar messages [ 825.640402] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 831.655267] Lustre: DEBUG MARKER: oleg635-server.virtnet: executing set_default_debug -1 all [ 839.365572] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 843.197396] Lustre: 37878:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify general/lod.*.mdt_hash=crush (mode = 0) failed: rc = -17 [ 849.890557] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 849.909829] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 849.955025] Lustre: Skipped 1 previous similar message [ 849.967227] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 849.978395] Lustre: Skipped 4 previous similar messages [ 854.810085] Lustre: server umount lustre-MDT0000 complete [ 859.907248] LustreError: 34835:0:(ldlm_lockd.c:2580:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1787191133 with bad export cookie 2193551412158369131 [ 859.932909] LustreError: MGC192.168.206.135@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 859.933134] LustreError: 34835:0:(ldlm_lockd.c:2580:ldlm_cancel_handler()) Skipped 3 previous similar messages [ 860.147055] Lustre: server umount lustre-MDT0001 complete [ 875.025357] Lustre: server umount lustre-OST0000 complete [ 877.857957] Lustre: 3648:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1787191135/real 1787191135] req@ffff968d3de91f80 x1874004905613312/t0(0) o400->lustre-MDT0001-lwp-OST0001@0@lo:12/10 lens 224/224 e 0 to 1 dl 1787191151 ref 1 fl Rpc:XNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 [ 879.249843] Lustre: server umount lustre-OST0001 complete [ 888.266394] Lustre: DEBUG MARKER: == conf-sanity test 76c: verify changelog_mask is applied with lctl set_param -P ========================================================== 21:59:19 (1787191159) [ 904.822667] Lustre: DEBUG MARKER: oleg635-server.virtnet: executing load_modules_local [ 914.065441] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 918.550393] Lustre: DEBUG MARKER: oleg635-server.virtnet: executing set_default_debug -1 all [ 926.505046] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 930.768843] Lustre: DEBUG MARKER: oleg635-server.virtnet: executing set_default_debug -1 all [ 933.343660] Lustre: 41171:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify general/debug_raw_pointers=Y (mode = 0) failed: rc = -17 [ 938.613853] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 944.347406] Lustre: DEBUG MARKER: oleg635-server.virtnet: executing set_default_debug -1 all [ 952.112949] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 952.383318] Lustre: lustre-OST0001: Imperative Recovery not enabled, recovery window 60-180 [ 952.387339] Lustre: Skipped 8 previous similar messages [ 957.277247] Lustre: DEBUG MARKER: oleg635-server.virtnet: executing set_default_debug -1 all [ 963.515340] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 966.244984] Lustre: 43047:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify general/lod.*.mdt_hash=crush (mode = 0) failed: rc = -17 [ 966.929284] Lustre: Setting parameter general.mdd.*.changelog_mask=-CLOSE in log params [ 966.934315] Lustre: Skipped 1 previous similar message [ 972.774320] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 975.234488] Lustre: server umount lustre-MDT0000 complete [ 978.411920] LustreError: 40005:0:(ldlm_lockd.c:2580:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1787191252 with bad export cookie 2193551412158370237 [ 978.413719] LustreError: MGC192.168.206.135@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 978.420923] LustreError: 40005:0:(ldlm_lockd.c:2580:ldlm_cancel_handler()) Skipped 2 previous similar messages [ 983.011368] Lustre: lustre-MDT0001-lwp-OST0001: Connection to lustre-MDT0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 983.022644] Lustre: Skipped 9 previous similar messages [ 984.618580] Lustre: server umount lustre-MDT0001 complete [ 992.848266] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1003.432292] Lustre: Evicted from MGS (at 0@lo) after server handle changed from 0x0 to 0x1e710f632e419624 [ 1003.451673] Lustre: MGC192.168.206.135@tcp: Connection restored to 0@lo (at 0@lo) [ 1007.637983] Lustre: DEBUG MARKER: oleg635-server.virtnet: executing set_default_debug -1 all [ 1012.912376] Lustre: lustre-MDT0000-lwp-OST0001: Connection restored to 0@lo (at 0@lo) [ 1014.763613] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1018.619793] Lustre: DEBUG MARKER: oleg635-server.virtnet: executing set_default_debug -1 all [ 1022.450222] Lustre: lustre-MDT0001-lwp-OST0000: Connection restored to 0@lo (at 0@lo) [ 1022.453368] Lustre: Skipped 1 previous similar message [ 1025.388159] Lustre: DEBUG MARKER: oleg635-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 1026.834803] Lustre: DEBUG MARKER: mdc.lustre-MDT0000-mdc-*.mds_server_uuid in FULL state after 0 sec [ 1031.945417] Lustre: DEBUG MARKER: oleg635-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 1033.370186] Lustre: DEBUG MARKER: mdc.lustre-MDT0001-mdc-*.mds_server_uuid in FULL state after 0 sec [ 1040.874449] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 1044.163296] Lustre: server umount lustre-MDT0000 complete [ 1047.356477] LustreError: 40003:0:(ldlm_lockd.c:2580:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1787191321 with bad export cookie 2193551412158371364 [ 1047.361408] LustreError: MGC192.168.206.135@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 1047.378955] LustreError: 40003:0:(ldlm_lockd.c:2580:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 1047.513789] Lustre: server umount lustre-MDT0001 complete [ 1061.091190] Lustre: server umount lustre-OST0000 complete [ 1063.375092] Lustre: 3650:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1787191321/real 1787191321] req@ffff968d3dfa8380 x1874004905694208/t0(0) o400->lustre-MDT0001-lwp-OST0001@0@lo:12/10 lens 224/224 e 0 to 1 dl 1787191337 ref 1 fl Rpc:XNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 [ 1063.799772] Lustre: server umount lustre-OST0001 complete [ 1070.746346] Lustre: DEBUG MARKER: == conf-sanity test 76d: verify llite.*.xattr_cache can be set by 'lctl set_param -P' correctly ========================================================== 22:02:22 (1787191342) [ 1085.711972] Lustre: DEBUG MARKER: oleg635-server.virtnet: executing load_modules_local [ 1094.800135] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1095.048437] LustreError: 47667:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 1095.063422] LustreError: 47667:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 22 previous similar messages [ 1099.151635] Lustre: DEBUG MARKER: oleg635-server.virtnet: executing set_default_debug -1 all [ 1108.148727] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1112.817788] Lustre: DEBUG MARKER: oleg635-server.virtnet: executing set_default_debug -1 all [ 1116.228318] Lustre: 48819:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify general/debug_raw_pointers=Y (mode = 0) failed: rc = -17 [ 1123.132869] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 1130.023886] Lustre: DEBUG MARKER: oleg635-server.virtnet: executing set_default_debug -1 all [ 1138.212653] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 1143.626675] Lustre: DEBUG MARKER: oleg635-server.virtnet: executing set_default_debug -1 all [ 1150.249570] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 1154.483590] Lustre: Setting parameter general.llite.*.xattr_cache=0 in log params [ 1164.769475] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 1164.784409] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 1164.795083] Lustre: Skipped 18 previous similar messages [ 1166.596234] Lustre: server umount lustre-MDT0000 complete [ 1169.580434] LustreError: 47648:0:(ldlm_lockd.c:2580:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1787191443 with bad export cookie 2193551412158372148 [ 1169.582342] LustreError: MGC192.168.206.135@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 1169.589491] LustreError: 47648:0:(ldlm_lockd.c:2580:ldlm_cancel_handler()) Skipped 1 previous similar message [ 1169.745374] Lustre: server umount lustre-MDT0001 complete [ 1182.990897] Lustre: server umount lustre-OST0000 complete [ 1195.525992] Lustre: server umount lustre-OST0001 complete [ 1203.898715] Lustre: DEBUG MARKER: == conf-sanity test 77: comma-separated MGS NIDs and failover node NIDs ========================================================== 22:04:35 (1787191475) [ 1206.044086] Lustre: DEBUG MARKER: SKIP: conf-sanity test_77 mixed loopback and real device not working [ 1207.493147] Lustre: DEBUG MARKER: == conf-sanity test 78: run resize2fs on MDT and OST filesystems ========================================================== 22:04:39 (1787191479) [ 1216.940176] Lustre: DEBUG MARKER: oleg635-server.virtnet: executing set_hostid [ 1224.090238] Lustre: DEBUG MARKER: oleg635-server.virtnet: executing load_modules_local [ 1243.899324] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1244.172272] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 1244.177686] Lustre: lustre-MDT0000: new disk, initializing [ 1244.270579] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 1248.563820] Lustre: DEBUG MARKER: oleg635-server.virtnet: executing set_default_debug -1 all [ 1257.669689] Lustre: DEBUG MARKER: oleg635-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 1264.064260] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 1264.335629] Lustre: 54888:0:(osd_compat.c:1353:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 1265.684735] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:0:ost] [ 1265.719221] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x240000400 [ 1269.472781] Lustre: DEBUG MARKER: oleg635-server.virtnet: executing set_default_debug -1 all [ 1278.111817] Lustre: DEBUG MARKER: oleg635-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 1280.486516] Lustre: DEBUG MARKER: create test files [ 1311.713411] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 1311.718098] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 1311.727190] Lustre: Skipped 9 previous similar messages [ 1313.577967] Lustre: server umount lustre-OST0000 complete [ 1316.644937] Lustre: server umount lustre-MDT0000 complete [ 1324.460762] Lustre: DEBUG MARKER: oleg635-server.virtnet: executing unload_modules_local [ 1326.512341] Key type lgssc unregistered [ 1326.728706] LNet: 56521:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 1326.734257] LNetError: 56521:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 1326.750409] LNet: Removed LNI 192.168.206.135@tcp [ 1327.414169] Key type .llcrypt unregistered [ 1327.415758] Key type ._llcrypt unregistered [ 1351.507228] Key type ._llcrypt registered [ 1351.509551] Key type .llcrypt registered [ 1351.612342] Lustre: DEBUG MARKER: oleg635-server.virtnet: executing load_modules_local [ 1352.670380] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 1352.709248] alg: No test for adler32 (adler32-zlib) [ 1353.868453] Lustre: Lustre: Build Version: 2.17.57_44_gafaaa93 [ 1354.136344] LNet: Added LNI 192.168.206.135@tcp [8/256/0/180] [ 1355.848230] Key type lgssc registered [ 1357.083849] Lustre: Echo OBD driver; http://www.lustre.org/ [ 1364.745480] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 1364.767111] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1366.169847] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 1370.074900] Lustre: DEBUG MARKER: oleg635-server.virtnet: executing set_default_debug -1 all [ 1376.664586] Lustre: DEBUG MARKER: oleg635-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 1383.430842] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 1383.720378] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 1388.709530] Lustre: DEBUG MARKER: oleg635-server.virtnet: executing set_default_debug -1 all [ 1390.897008] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x240000400:102 to 0x240000400:129) [ 1390.981364] Lustre: 59233:0:(osd_internal.h:1460:osd_trans_exec_op()) lustre-OST0000: opcode 8: before 510 < left 544, rollback = 9 [ 1390.994200] Lustre: 59233:0:(osd_handler.c:2090:osd_trans_dump_creds()) create: 32/128/5, destroy: 0/0/0 [ 1390.999814] Lustre: 59233:0:(osd_handler.c:2097:osd_trans_dump_creds()) attr_set: 0/0/0, xattr_set: 0/0/0 [ 1391.003923] Lustre: 59233:0:(osd_handler.c:2107:osd_trans_dump_creds()) write: 2/18/1, punch: 0/0/0, quota 1/3/0 [ 1391.014943] Lustre: 59233:0:(osd_handler.c:2114:osd_trans_dump_creds()) insert: 32/544/0, delete: 0/0/0 [ 1391.020114] Lustre: 59233:0:(osd_handler.c:2121:osd_trans_dump_creds()) ref_add: 0/0/0, ref_del: 0/0/0 [ 1394.935946] Lustre: DEBUG MARKER: oleg635-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 1399.846438] Lustre: DEBUG MARKER: check files after expanding the MDT and OST filesystems [ 1404.510738] Lustre: DEBUG MARKER: create more files after expanding the MDT and OST filesystems [ 1411.432834] Lustre: server umount lustre-OST0000 complete [ 1425.002784] Lustre: server umount lustre-MDT0000 complete [ 1434.835324] Lustre: DEBUG MARKER: oleg635-server.virtnet: executing unload_modules_local [ 1437.464793] Key type lgssc unregistered [ 1437.676840] LNet: 60848:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 1437.682661] LNetError: 60848:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 1437.698584] LNet: Removed LNI 192.168.206.135@tcp [ 1438.414630] Key type .llcrypt unregistered [ 1438.416323] Key type ._llcrypt unregistered [ 1461.961863] Key type ._llcrypt registered [ 1461.963678] Key type .llcrypt registered [ 1462.080233] Lustre: DEBUG MARKER: oleg635-server.virtnet: executing load_modules_local [ 1462.941829] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 1462.963554] alg: No test for adler32 (adler32-zlib) [ 1463.996910] Lustre: Lustre: Build Version: 2.17.57_44_gafaaa93 [ 1464.265268] LNet: Added LNI 192.168.206.135@tcp [8/256/0/180] [ 1465.919190] Key type lgssc registered [ 1467.240094] Lustre: Echo OBD driver; http://www.lustre.org/ [ 1475.275824] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 1475.316450] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1476.702909] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 1482.416366] Lustre: DEBUG MARKER: oleg635-server.virtnet: executing set_default_debug -1 all [ 1489.305111] Lustre: DEBUG MARKER: oleg635-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 1495.332383] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 1495.730218] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 1501.879102] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x240000400:140 to 0x240000400:161) [ 1501.997642] Lustre: 63664:0:(osd_internal.h:1460:osd_trans_exec_op()) lustre-OST0000: opcode 8: before 514 < left 544, rollback = 9 [ 1502.016449] Lustre: 63664:0:(osd_handler.c:2090:osd_trans_dump_creds()) create: 32/128/1, destroy: 0/0/0 [ 1502.032221] Lustre: 63664:0:(osd_handler.c:2097:osd_trans_dump_creds()) attr_set: 0/0/0, xattr_set: 0/0/0 [ 1502.047642] Lustre: 63664:0:(osd_handler.c:2107:osd_trans_dump_creds()) write: 2/18/1, punch: 0/0/0, quota 1/3/0 [ 1502.072681] Lustre: 63664:0:(osd_handler.c:2114:osd_trans_dump_creds()) insert: 32/544/0, delete: 0/0/0 [ 1502.082385] Lustre: 63664:0:(osd_handler.c:2121:osd_trans_dump_creds()) ref_add: 0/0/0, ref_del: 0/0/0 [ 1502.370181] Lustre: DEBUG MARKER: oleg635-server.virtnet: executing set_default_debug -1 all [ 1510.309374] Lustre: DEBUG MARKER: oleg635-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 1516.666865] Lustre: DEBUG MARKER: check files after shrinking the MDT and OST filesystems [ 1527.778209] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 1527.784602] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 1527.811242] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 1530.534380] Lustre: server umount lustre-OST0000 complete [ 1533.707989] Lustre: server umount lustre-MDT0000 complete [ 1540.663930] Lustre: DEBUG MARKER: oleg635-server.virtnet: executing unload_modules_local [ 1543.445346] Key type lgssc unregistered [ 1543.729796] LNet: 65224:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 1543.735330] LNetError: 65224:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 1543.752376] LNet: Removed LNI 192.168.206.135@tcp [ 1544.686643] Key type .llcrypt unregistered [ 1544.689242] Key type ._llcrypt unregistered [ 1561.079974] Key type ._llcrypt registered [ 1561.083232] Key type .llcrypt registered [ 1561.186707] Lustre: DEBUG MARKER: oleg635-server.virtnet: executing set_hostid [ 1573.638439] Lustre: DEBUG MARKER: oleg635-server.virtnet: executing load_modules_local [ 1574.424041] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 1574.534181] alg: No test for adler32 (adler32-zlib) [ 1575.537466] Lustre: Lustre: Build Version: 2.17.57_44_gafaaa93 [ 1575.728479] LNet: Added LNI 192.168.206.135@tcp [8/256/0/180] [ 1577.399308] Key type lgssc registered [ 1578.284300] Lustre: Echo OBD driver; http://www.lustre.org/ [ 1607.664080] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 1607.682420] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1608.875642] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 1608.909946] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 1608.914890] Lustre: lustre-MDT0000: new disk, initializing [ 1608.993470] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 1609.023147] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 1612.839876] Lustre: DEBUG MARKER: oleg635-server.virtnet: executing set_default_debug -1 all [ 1622.071946] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1622.199664] Lustre: 68355:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 1622.279787] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [ 1622.287559] Lustre: Skipped 1 previous similar message [ 1622.291388] Lustre: lustre-MDT0001: new disk, initializing [ 1622.399416] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 1622.430946] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [ 1622.436462] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 1626.254926] Lustre: DEBUG MARKER: oleg635-server.virtnet: executing set_default_debug -1 all [ 1634.166717] Lustre: DEBUG MARKER: oleg635-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 1638.716213] Lustre: DEBUG MARKER: oleg635-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 1644.987269] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 1645.230154] Lustre: lustre-OST0000: new disk, initializing [ 1645.234665] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 1645.242181] Lustre: 70324:0:(osd_compat.c:1353:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 1645.319553] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 1650.498777] Lustre: DEBUG MARKER: oleg635-server.virtnet: executing set_default_debug -1 all [ 1650.759586] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [ 1650.769204] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 1650.861823] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 1657.787639] Lustre: DEBUG MARKER: oleg635-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 1661.906990] Lustre: DEBUG MARKER: oleg635-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 [ 1662.081810] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [ 1666.118701] Lustre: DEBUG MARKER: oleg635-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid 50 [ 1666.315298] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [ 1671.653722] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 1671.655102] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 1671.670734] LustreError: Skipped 1 previous similar message [ 1671.672105] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 1671.704960] Lustre: Skipped 1 previous similar message [ 1676.770156] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 1676.776176] Lustre: Skipped 2 previous similar messages [ 1681.891857] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 1681.897515] Lustre: Skipped 1 previous similar message [ 1682.399207] Lustre: lustre-OST0000 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 1682.589868] Lustre: server umount lustre-OST0000 complete [ 1687.018610] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 1687.030608] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 1692.130462] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 1692.136898] Lustre: Skipped 2 previous similar messages [ 1692.305216] Lustre: server umount lustre-MDT0000 complete [ 1696.236445] LustreError: 68350:0:(ldlm_lockd.c:2580:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1787191969 with bad export cookie 16876731856266272419 [ 1696.237441] LustreError: MGC192.168.206.135@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 1696.246818] LustreError: 68350:0:(ldlm_lockd.c:2580:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 1696.392627] Lustre: server umount lustre-MDT0001 complete [ 1705.248280] Lustre: DEBUG MARKER: == conf-sanity test 79: format MDT/OST without mgs option (should return errors) ========================================================== 22:12:57 (1787191977) [ 1717.379471] Lustre: DEBUG MARKER: oleg635-server.virtnet: executing load_modules_local [ 1756.898780] Lustre: DEBUG MARKER: oleg635-server.virtnet: executing set_hostid [ 1769.478916] Lustre: DEBUG MARKER: oleg635-server.virtnet: executing load_modules_local [ 1806.744255] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1807.005125] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 1807.053607] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 1807.059764] Lustre: lustre-MDT0000: new disk, initializing [ 1807.154869] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 1807.183522] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 1811.592112] Lustre: DEBUG MARKER: oleg635-server.virtnet: executing set_default_debug -1 all [ 1823.304951] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1823.450588] Lustre: 76344:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 1823.509686] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [ 1823.514324] Lustre: Skipped 1 previous similar message [ 1823.517326] Lustre: lustre-MDT0001: new disk, initializing [ 1823.642621] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 1823.693526] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [ 1823.704785] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 1828.261980] Lustre: DEBUG MARKER: oleg635-server.virtnet: executing set_default_debug -1 all [ 1838.967581] Lustre: DEBUG MARKER: oleg635-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 1844.735226] Lustre: DEBUG MARKER: oleg635-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 1851.213176] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 1851.452203] Lustre: lustre-OST0000: new disk, initializing [ 1851.459775] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 1851.465394] Lustre: 78315:0:(osd_compat.c:1353:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 1851.559934] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 1858.321667] Lustre: DEBUG MARKER: oleg635-server.virtnet: executing set_default_debug -1 all [ 1858.680317] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [ 1858.697595] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 1858.851411] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 1868.206467] Lustre: DEBUG MARKER: oleg635-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 1872.672505] Lustre: DEBUG MARKER: oleg635-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 [ 1872.920161] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [ 1876.960628] Lustre: DEBUG MARKER: oleg635-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid 50 [ 1877.123987] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [ 1879.523043] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 1879.529738] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 1879.539446] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 1879.555471] Lustre: Skipped 2 previous similar messages [ 1893.345545] Lustre: lustre-OST0000 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 1893.469386] Lustre: server umount lustre-OST0000 complete [ 1900.001925] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 1900.011424] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 1900.028918] Lustre: Skipped 1 previous similar message [ 1900.045575] Lustre: Skipped 6 previous similar messages [ 1903.357633] Lustre: server umount lustre-MDT0000 complete [ 1906.951070] LustreError: 77138:0:(ldlm_lockd.c:2580:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1787192180 with bad export cookie 16876731856266273301 [ 1906.961558] LustreError: 77138:0:(ldlm_lockd.c:2580:ldlm_cancel_handler()) Skipped 1 previous similar message [ 1906.964158] LustreError: MGC192.168.206.135@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 1907.152835] Lustre: server umount lustre-MDT0001 complete [ 1914.916814] Lustre: DEBUG MARKER: == conf-sanity test 80: mgc import reconnect race ======== 22:16:26 (1787192186) [ 1922.345347] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1922.859402] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 1927.534610] Lustre: DEBUG MARKER: oleg635-server.virtnet: executing set_default_debug -1 all [ 1928.169276] LustreError: 80727:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 1928.196191] LustreError: 80727:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 2 previous similar messages [ 1933.284574] LustreError: 80726:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 1937.032908] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1942.054989] Lustre: DEBUG MARKER: oleg635-server.virtnet: executing set_default_debug -1 all [ 1951.097292] Lustre: DEBUG MARKER: oleg635-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 1957.038654] Lustre: DEBUG MARKER: oleg635-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 1963.716236] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 1964.101953] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 1964.107375] Lustre: Skipped 1 previous similar message [ 1969.662671] Lustre: DEBUG MARKER: oleg635-server.virtnet: executing set_default_debug -1 all [ 1977.139272] Lustre: DEBUG MARKER: oleg635-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 1980.499233] Lustre: 83398:0:(genops.c:1774:obd_export_evict_by_uuid()) MGS: evicting 2659af45-7a3c-4032-96af-0927b8f550fa at adminstrative request [ 1981.477306] LustreError: MGC192.168.206.135@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 1981.492074] Lustre: *** cfs_fail_loc=906, val=2147483648*** [ 1991.651918] Lustre: MGS: Client 2659af45-7a3c-4032-96af-0927b8f550fa (at 0@lo) reconnecting [ 1991.667440] Lustre: Evicted from MGS (at 0@lo) after server handle changed from 0xea363234de452587 to 0xea363234de452827 [ 1991.690067] Lustre: MGC192.168.206.135@tcp: Connection restored to 0@lo (at 0@lo) [ 2018.153034] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 2018.378829] Lustre: lustre-OST0001: new disk, initializing [ 2018.381935] Lustre: srv-lustre-OST0001: No data found on store. Initialize space. [ 2018.386694] Lustre: 83792:0:(osd_compat.c:1353:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 2024.985753] Lustre: DEBUG MARKER: oleg635-server.virtnet: executing set_default_debug -1 all [ 2025.049453] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x00000002c0000400-0x0000000300000400]:1:ost [ 2025.062277] Lustre: cli-lustre-OST0001-super: Allocated super-sequence [0x00000002c0000400-0x0000000300000400]:1:ost] [ 2025.169384] Lustre: lustre-OST0001-osc-MDT0000: update sequence from 0x100010000 to 0x2c0000401 [ 2033.586301] Lustre: DEBUG MARKER: oleg635-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0001-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 2040.813570] Lustre: lustre-OST0001-osc-MDT0000: Connection to lustre-OST0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2040.819281] LustreError: lustre-OST0001-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 2040.833980] Lustre: lustre-OST0001: Not available for connect from 0@lo (stopping) [ 2040.833992] Lustre: Skipped 2 previous similar messages [ 2040.836717] Lustre: Skipped 1 previous similar message [ 2040.842632] LustreError: Skipped 2 previous similar messages [ 2042.289920] Lustre: server umount lustre-OST0001 complete [ 2045.947884] LustreError: 82551:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 2045.978767] LustreError: 82551:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 1 previous similar message [ 2047.975252] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 2051.040243] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2051.067404] Lustre: Skipped 1 previous similar message [ 2051.098662] LustreError: 82549:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 2051.116824] LustreError: 82549:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 1 previous similar message [ 2052.592561] Lustre: server umount lustre-OST0000 complete [ 2060.260068] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 2060.268702] LustreError: Skipped 1 previous similar message [ 2063.309082] Lustre: server umount lustre-MDT0000 complete [ 2066.402370] LustreError: 83411:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 2066.418544] LustreError: 83411:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 1 previous similar message [ 2067.408223] LustreError: 81462:0:(ldlm_lockd.c:2580:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1787192341 with bad export cookie 16876731856266274855 [ 2067.418717] LustreError: MGC192.168.206.135@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 2067.433633] LustreError: 81462:0:(ldlm_lockd.c:2580:ldlm_cancel_handler()) Skipped 2 previous similar messages [ 2067.755261] Lustre: server umount lustre-MDT0001 complete [ 2076.387483] Lustre: DEBUG MARKER: == conf-sanity test 81: sparse OST indexing ============== 22:19:08 (1787192348) [ 2078.408506] Lustre: DEBUG MARKER: SKIP: conf-sanity test_81 needs >= 3 OSTs [ 2080.633767] Lustre: DEBUG MARKER: == conf-sanity test 82a: specify OSTs for file (succeed) or directory (succeed) ========================================================== 22:19:11 (1787192351) [ 2082.434592] Lustre: DEBUG MARKER: SKIP: conf-sanity test_82a needs >= 3 OSTs [ 2084.320527] Lustre: DEBUG MARKER: == conf-sanity test 82b: specify OSTs for file with --pool and --ost-list options ========================================================== 22:19:15 (1787192355) [ 2086.023509] Lustre: DEBUG MARKER: SKIP: conf-sanity test_82b needs >= 4 OSTs [ 2087.579498] Lustre: DEBUG MARKER: == conf-sanity test 82c: specify sparse OSTs for setstripe (should not crash) ========================================================== 22:19:19 (1787192359) [ 2089.051750] Lustre: DEBUG MARKER: SKIP: conf-sanity test_82c needs >= 4 OSTs [ 2091.056360] Lustre: DEBUG MARKER: == conf-sanity test 83: ENOSPACE on OST doesn't cause message VFS: Busy inodes after unmount ... ========================================================== 22:19:22 (1787192362) [ 2092.917869] Lustre: DEBUG MARKER: mount the OST /dev/mapper/ost1_flakey as a ldiskfs filesystem [ 2099.155297] loop: module loaded [ 2099.175669] loop0: detected capacity change from 0 to 4294967296 [ 2099.264537] LDISKFS-fs (loop0): mounted filesystem with ordered data mode. Opts: (null) [ 2100.862484] Lustre: DEBUG MARKER: run llverfs in partial mode on the OST ldiskfs /mnt/lustre-ost1 [ 2104.558561] Lustre: DEBUG MARKER: oleg635-server.virtnet: executing run_llverfs /mnt/lustre-ost1 -vpl no [ 2108.822571] Lustre: DEBUG MARKER: unmount the OST /dev/mapper/ost1_flakey [ 2134.274966] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 2134.316978] LustreError: 87918:0:(obd_config.c:845:class_setup()) setup lustre-OST0000-osd failed (-28) [ 2134.327662] LustreError: 87918:0:(obd_mount.c:259:lustre_start_simple()) lustre-OST0000-osd setup error -28 [ 2134.333749] LustreError: 87918:0:(tgt_mount.c:2579:server_fill_super()) Unable to start osd on /dev/mapper/ost1_flakey: -28 [ 2134.339254] LustreError: 87918:0:(super25.c:179:lustre_fill_super()) llite: Unable to mount : rc = -28 [ 2153.091174] Lustre: DEBUG MARKER: oleg635-server.virtnet: executing set_hostid [ 2163.865388] Lustre: DEBUG MARKER: oleg635-server.virtnet: executing load_modules_local [ 2207.682899] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 2207.897756] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 2207.924323] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 2207.929533] Lustre: lustre-MDT0000: new disk, initializing [ 2207.993736] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 2207.999068] Lustre: Skipped 1 previous similar message [ 2208.023499] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 2213.555376] Lustre: DEBUG MARKER: oleg635-server.virtnet: executing set_default_debug -1 all [ 2224.917958] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 2225.094729] Lustre: 90783:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 2225.234144] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 2230.172606] Lustre: DEBUG MARKER: oleg635-server.virtnet: executing set_default_debug -1 all [ 2241.724909] Lustre: DEBUG MARKER: oleg635-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 2248.448512] Lustre: DEBUG MARKER: oleg635-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 2256.457106] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 2257.006759] Lustre: 92753:0:(osd_compat.c:1353:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 2263.177726] Lustre: DEBUG MARKER: oleg635-server.virtnet: executing set_default_debug -1 all [ 2264.133380] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 2264.277744] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 2272.370380] Lustre: DEBUG MARKER: oleg635-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 2277.646275] Lustre: DEBUG MARKER: oleg635-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 [ 2277.831460] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [ 2282.526689] Lustre: DEBUG MARKER: oleg635-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid 50 [ 2282.776672] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [ 2285.025610] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2285.029223] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 2285.047470] Lustre: Skipped 2 previous similar messages [ 2285.051426] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 2285.056180] LustreError: Skipped 1 previous similar message [ 2285.061222] Lustre: Skipped 8 previous similar messages [ 2290.703910] Lustre: server umount lustre-OST0000 complete [ 2300.356591] Lustre: server umount lustre-MDT0000 complete [ 2300.392053] LustreError: 90796:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 2300.418750] LustreError: 90796:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 1 previous similar message [ 2304.088595] LustreError: 90775:0:(ldlm_lockd.c:2580:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1787192577 with bad export cookie 16876731856266275471 [ 2304.089312] LustreError: MGC192.168.206.135@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 2304.093664] LustreError: 90775:0:(ldlm_lockd.c:2580:ldlm_cancel_handler()) Skipped 2 previous similar messages [ 2304.266203] Lustre: server umount lustre-MDT0001 complete [ 2314.026986] Lustre: DEBUG MARKER: == conf-sanity test 84: check recovery_hard_time ========= 22:23:05 (1787192585) [ 2320.947891] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 2321.312896] LustreError: 95165:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 2325.092103] Lustre: DEBUG MARKER: oleg635-server.virtnet: executing set_default_debug -1 all [ 2332.091982] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 2335.616112] Lustre: DEBUG MARKER: oleg635-server.virtnet: executing set_default_debug -1 all [ 2342.155920] Lustre: DEBUG MARKER: oleg635-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 2347.773528] Lustre: DEBUG MARKER: oleg635-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 2355.266391] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 2355.711321] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 2355.722934] Lustre: Skipped 4 previous similar messages [ 2362.716446] Lustre: DEBUG MARKER: oleg635-server.virtnet: executing set_default_debug -1 all [ 2370.730709] Lustre: DEBUG MARKER: oleg635-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 2376.182621] Lustre: DEBUG MARKER: oleg635-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 [ 2376.437858] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [ 2381.935842] Lustre: DEBUG MARKER: oleg635-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid 50 [ 2382.216377] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [ 2389.689209] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 2389.830788] Lustre: lustre-OST0001: new disk, initializing [ 2389.834167] Lustre: Skipped 2 previous similar messages [ 2389.838810] Lustre: srv-lustre-OST0001: No data found on store. Initialize space. [ 2389.843273] Lustre: Skipped 3 previous similar messages [ 2389.848864] Lustre: 98536:0:(osd_compat.c:1353:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 2396.875496] Lustre: DEBUG MARKER: oleg635-server.virtnet: executing set_default_debug -1 all [ 2400.392024] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x00000002c0000400-0x0000000300000400]:1:ost [ 2400.402570] Lustre: Skipped 2 previous similar messages [ 2400.410373] Lustre: cli-lustre-OST0001-super: Allocated super-sequence [0x00000002c0000400-0x0000000300000400]:1:ost] [ 2400.537482] Lustre: lustre-OST0001-osc-MDT0000: update sequence from 0x100010000 to 0x2c0000401 [ 2407.106337] Lustre: DEBUG MARKER: oleg635-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0001-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 2412.757412] Lustre: DEBUG MARKER: oleg635-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0001-osc-MDT0000.ost_server_uuid 50 [ 2412.992720] Lustre: DEBUG MARKER: os[cp].lustre-OST0001-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [ 2418.797159] Lustre: DEBUG MARKER: oleg635-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0001-osc-MDT0001.ost_server_uuid 50 [ 2419.029889] Lustre: DEBUG MARKER: os[cp].lustre-OST0001-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [ 2438.501722] Lustre: DEBUG MARKER: mds1 REPLAY BARRIER on lustre-MDT0000 [ 2462.979155] Lustre: Failing over lustre-MDT0000 [ 2463.287940] Lustre: server umount lustre-MDT0000 complete [ 2465.760686] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 2465.767351] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2465.783620] Lustre: Skipped 3 previous similar messages [ 2465.787073] LustreError: 95894:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 2465.806241] LustreError: 95894:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 2 previous similar messages [ 2483.679607] Lustre: 66582:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1787192741/real 1787192741] req@00000000b93ddba4 x1874006480530816/t0(0) o400->MGC192.168.206.135@tcp@0@lo:26/25 lens 224/224 e 0 to 1 dl 1787192757 ref 1 fl Rpc:XNQr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 [ 2483.759300] LustreError: MGC192.168.206.135@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 2492.929439] LDISKFS-fs (dm-0): 4 truncates cleaned up [ 2492.934441] LDISKFS-fs (dm-0): recovery complete [ 2492.943742] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 2494.175946] LustreError: 101356:0:(import.c:339:ptlrpc_invalidate_import()) MGS: timeout waiting for callback (1 != 0) [ 2494.182049] LustreError: 101356:0:(import.c:363:ptlrpc_invalidate_import()) @@@ still on sending list req@000000005ffc8fec x1874006480536064/t0(0) o250->MGC192.168.206.135@tcp@0@lo:26/25 lens 520/544 e 0 to 0 dl 1787192767 ref 1 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 [ 2494.248734] LustreError: 101356:0:(import.c:373:ptlrpc_invalidate_import()) MGS: Unregistering RPCs found (0). Network is sluggish? Waiting for them to error out. [ 2494.946591] LustreError: 66578:0:(client.c:1404:ptlrpc_import_delay_req()) @@@ invalidate in flight req@0000000038932ebc x1874006480539776/t0(0) o250->MGC192.168.206.135@tcp@0@lo:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl Rpc:NQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 [ 2495.193992] Lustre: lustre-MDT0000: in recovery but waiting for the first client to connect [ 2499.102552] Lustre: lustre-MDT0000: Will be in recovery for at least 1:00, or until 3 clients reconnect [ 2499.115695] Lustre: 95160:0:(ldlm_lib.c:2110:extend_recovery_timer()) lustre-MDT0000: extended recovery timer reached hard limit: 60, extend: 0 [ 2500.611310] Lustre: lustre-MDT0000-lwp-MDT0001: Connection restored to 0@lo (at 0@lo) [ 2500.613426] Lustre: 95165:0:(ldlm_lib.c:2110:extend_recovery_timer()) lustre-MDT0000: extended recovery timer reached hard limit: 60, extend: 0 [ 2500.631927] Lustre: 95165:0:(ldlm_lib.c:2110:extend_recovery_timer()) Skipped 1 previous similar message [ 2500.736488] LustreError: 101388:0:(ldlm_lib.c:2723:replay_request_or_update()) cfs_fail_timeout id 709 sleeping for 300ms [ 2501.059290] LustreError: 101388:0:(ldlm_lib.c:2723:replay_request_or_update()) cfs_fail_timeout id 709 awake [ 2501.333462] Lustre: DEBUG MARKER: oleg635-server.virtnet: executing set_default_debug -1 all [ 2501.393600] LustreError: 101388:0:(ldlm_lib.c:2723:replay_request_or_update()) cfs_fail_timeout id 709 sleeping for 300ms [ 2501.412704] LustreError: 101388:0:(ldlm_lib.c:2723:replay_request_or_update()) Skipped 1 previous similar message [ 2501.735210] LustreError: 101388:0:(ldlm_lib.c:2723:replay_request_or_update()) cfs_fail_timeout id 709 awake [ 2501.742075] LustreError: 101388:0:(ldlm_lib.c:2723:replay_request_or_update()) Skipped 1 previous similar message [ 2501.758642] Lustre: 101388:0:(ldlm_lib.c:2110:extend_recovery_timer()) lustre-MDT0000: extended recovery timer reached hard limit: 60, extend: 1 [ 2501.777415] Lustre: 101388:0:(ldlm_lib.c:2110:extend_recovery_timer()) Skipped 7 previous similar messages [ 2502.440715] LustreError: 101388:0:(ldlm_lib.c:2723:replay_request_or_update()) cfs_fail_timeout id 709 sleeping for 300ms [ 2502.466862] LustreError: 101388:0:(ldlm_lib.c:2723:replay_request_or_update()) Skipped 2 previous similar messages [ 2502.788305] LustreError: 101388:0:(ldlm_lib.c:2723:replay_request_or_update()) cfs_fail_timeout id 709 awake [ 2502.802307] LustreError: 101388:0:(ldlm_lib.c:2723:replay_request_or_update()) Skipped 2 previous similar messages [ 2503.826876] Lustre: 101388:0:(ldlm_lib.c:2110:extend_recovery_timer()) lustre-MDT0000: extended recovery timer reached hard limit: 60, extend: 1 [ 2503.847325] Lustre: 101388:0:(ldlm_lib.c:2110:extend_recovery_timer()) Skipped 5 previous similar messages [ 2504.504604] LustreError: 101388:0:(ldlm_lib.c:2723:replay_request_or_update()) cfs_fail_timeout id 709 sleeping for 300ms [ 2504.534492] LustreError: 101388:0:(ldlm_lib.c:2723:replay_request_or_update()) Skipped 5 previous similar messages [ 2504.864071] LustreError: 101388:0:(ldlm_lib.c:2723:replay_request_or_update()) cfs_fail_timeout id 709 awake [ 2504.885415] LustreError: 101388:0:(ldlm_lib.c:2723:replay_request_or_update()) Skipped 5 previous similar messages [ 2508.073289] Lustre: 101388:0:(ldlm_lib.c:2110:extend_recovery_timer()) lustre-MDT0000: extended recovery timer reached hard limit: 60, extend: 1 [ 2508.099516] Lustre: 101388:0:(ldlm_lib.c:2110:extend_recovery_timer()) Skipped 12 previous similar messages [ 2508.752190] LustreError: 101388:0:(ldlm_lib.c:2723:replay_request_or_update()) cfs_fail_timeout id 709 sleeping for 300ms [ 2508.774547] LustreError: 101388:0:(ldlm_lib.c:2723:replay_request_or_update()) Skipped 12 previous similar messages [ 2509.095485] LustreError: 101388:0:(ldlm_lib.c:2723:replay_request_or_update()) cfs_fail_timeout id 709 awake [ 2509.102210] LustreError: 101388:0:(ldlm_lib.c:2723:replay_request_or_update()) Skipped 12 previous similar messages [ 2516.104357] Lustre: 101388:0:(ldlm_lib.c:2110:extend_recovery_timer()) lustre-MDT0000: extended recovery timer reached hard limit: 60, extend: 1 [ 2516.121879] Lustre: 101388:0:(ldlm_lib.c:2110:extend_recovery_timer()) Skipped 24 previous similar messages [ 2516.768519] LustreError: 101388:0:(ldlm_lib.c:2723:replay_request_or_update()) cfs_fail_timeout id 709 sleeping for 300ms [ 2516.785718] LustreError: 101388:0:(ldlm_lib.c:2723:replay_request_or_update()) Skipped 24 previous similar messages [ 2517.103120] LustreError: 101388:0:(ldlm_lib.c:2723:replay_request_or_update()) cfs_fail_timeout id 709 awake [ 2517.116638] LustreError: 101388:0:(ldlm_lib.c:2723:replay_request_or_update()) Skipped 24 previous similar messages [ 2532.386298] Lustre: 101388:0:(ldlm_lib.c:2110:extend_recovery_timer()) lustre-MDT0000: extended recovery timer reached hard limit: 60, extend: 1 [ 2532.406675] Lustre: 101388:0:(ldlm_lib.c:2110:extend_recovery_timer()) Skipped 50 previous similar messages [ 2533.057586] LustreError: 101388:0:(ldlm_lib.c:2723:replay_request_or_update()) cfs_fail_timeout id 709 sleeping for 300ms [ 2533.079509] LustreError: 101388:0:(ldlm_lib.c:2723:replay_request_or_update()) Skipped 50 previous similar messages [ 2533.400114] LustreError: 101388:0:(ldlm_lib.c:2723:replay_request_or_update()) cfs_fail_timeout id 709 awake [ 2533.412364] LustreError: 101388:0:(ldlm_lib.c:2723:replay_request_or_update()) Skipped 50 previous similar messages [ 2559.175137] Lustre: 101388:0:(ldlm_lib.c:2419:target_recovery_overseer()) lustre-MDT0000 recovery is aborted by hard timeout [ 2559.184945] Lustre: 101388:0:(ldlm_lib.c:2429:target_recovery_overseer()) recovery is aborted, evict exports in recovery [ 2559.200666] Lustre: 101388:0:(genops.c:1601:class_disconnect_stale_exports()) lustre-MDT0000: disconnect stale client f459ba4f-7118-473e-b7f1-1717e9f097b6@192.168.206.35@tcp [ 2559.217180] Lustre: lustre-MDT0000: disconnecting 1 stale clients [ 2559.229664] Lustre: 101388:0:(ldlm_lib.c:1939:abort_req_replay_queue()) @@@ aborted: req@000000008ca73064 x1874006466697600/t0(8589934779) o101->f459ba4f-7118-473e-b7f1-1717e9f097b6@192.168.206.35@tcp:633/0 lens 592/0 e 0 to 0 dl 1787192843 ref 1 fl Complete:/604/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:0 [ 2559.258687] Lustre: lustre-MDT0000-osd: cancel update llog [0x200000400:0x1:0x0] [ 2559.270389] Lustre: lustre-MDT0000: Denying connection for new client f459ba4f-7118-473e-b7f1-1717e9f097b6 (at 192.168.206.35@tcp), waiting for 3 known clients (2 recovered, 0 in progress, and 1 evicted) to recover in 0:00 [ 2559.305170] Lustre: lustre-MDT0001-osp-MDT0000: cancel update llog [0x240000401:0x1:0x0] [ 2559.351954] Lustre: lustre-MDT0000: Recovery over after 1:00, of 3 clients 2 recovered and 1 was evicted. [ 2559.356675] Lustre: lustre-MDT0000-osp-MDT0001: Connection restored to 0@lo (at 0@lo) [ 2559.369068] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:79 to 0x280000401:577) [ 2559.369937] Lustre: lustre-OST0001: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x2c0000401:48 to 0x2c0000401:545) [ 2559.374603] Lustre: Skipped 2 previous similar messages [ 2575.844670] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 2575.865236] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2575.900182] Lustre: Skipped 3 previous similar messages [ 2575.910527] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 2575.917829] Lustre: Skipped 4 previous similar messages [ 2579.737081] Lustre: server umount lustre-OST0000 complete [ 2582.513559] LustreError: 96989:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-OST0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 2582.530585] LustreError: 96989:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 47 previous similar messages [ 2598.369047] Lustre: lustre-OST0001 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 2598.610868] Lustre: server umount lustre-OST0001 complete [ 2611.564404] Lustre: server umount lustre-MDT0000 complete [ 2616.241633] LustreError: 95145:0:(ldlm_lockd.c:2580:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1787192889 with bad export cookie 16876731856266326690 [ 2616.245935] LustreError: MGC192.168.206.135@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 2616.247856] LustreError: 95145:0:(ldlm_lockd.c:2580:ldlm_cancel_handler()) Skipped 2 previous similar messages [ 2616.415637] Lustre: server umount lustre-MDT0001 complete [ 2629.290865] Lustre: DEBUG MARKER: == conf-sanity test 85: osd_ost init: fail ea_fid_set ==== 22:28:20 (1787192900) [ 2638.577295] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 2638.583344] Lustre: *** cfs_fail_loc=197, val=0*** [ 2654.180276] LustreError: Failed to get MGS log lustre-sptlrpc and no local copy. [ 2659.295451] LustreError: 103367:0:(mgc_request.c:233:do_config_log_add()) MGC192.168.206.135@tcp: failed processing log, type 4: rc = -110 [ 2664.436343] Lustre: *** cfs_fail_loc=197, val=0*** [ 2664.439500] Lustre: Skipped 65 previous similar messages [ 2685.023696] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 2685.029199] Lustre: Skipped 2 previous similar messages [ 2689.973270] Lustre: DEBUG MARKER: oleg635-server.virtnet: executing set_default_debug -1 all [ 2696.715198] Lustre: DEBUG MARKER: oleg635-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 2698.311775] Lustre: server umount lustre-OST0000 complete [ 2704.820671] Lustre: DEBUG MARKER: == conf-sanity test 86: Replacing mkfs.lustre -G option == 22:29:36 (1787192976) [ 2723.876856] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 2724.152746] LustreError: 104996:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 2724.162091] LustreError: 104996:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 8 previous similar messages [ 2727.807587] Lustre: DEBUG MARKER: oleg635-server.virtnet: executing set_default_debug -1 all [ 2735.763683] Lustre: DEBUG MARKER: oleg635-client.virtnet: executing wait_import_state_mount (FULL|IDLE) mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 2738.867887] LustreError: 105027:0:(osp_object.c:618:osp_attr_get()) lustre-MDT0001-osp-MDT0000: osp_attr_get update error [0x200000009:0x1:0x0]: rc = -5 [ 2738.876728] LustreError: 105027:0:(lod_sub_object.c:917:lod_sub_prep_llog()) lustre-MDT0000-mdtlov: can't get id from catalogs: rc = -5 [ 2738.883725] LustreError: 105027:0:(lod_dev.c:511:lod_sub_recovery_thread()) lustre-MDT0001-osp-MDT0000: get update log duration 15, retries 0, failed: rc = -5 [ 2738.994154] Lustre: server umount lustre-MDT0000 complete [ 2769.335762] Lustre: DEBUG MARKER: oleg635-server.virtnet: executing set_hostid [ 2778.859759] Lustre: DEBUG MARKER: oleg635-server.virtnet: executing load_modules_local [ 2813.688501] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 2813.866961] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 2813.886488] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 2813.892040] Lustre: lustre-MDT0000: new disk, initializing [ 2813.949609] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 2817.867862] Lustre: DEBUG MARKER: oleg635-server.virtnet: executing set_default_debug -1 all [ 2826.419846] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 2826.516315] Lustre: 109203:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 2826.696522] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 2830.540742] Lustre: DEBUG MARKER: oleg635-server.virtnet: executing set_default_debug -1 all [ 2838.826059] Lustre: DEBUG MARKER: oleg635-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 2844.012467] Lustre: DEBUG MARKER: oleg635-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 2850.064142] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 2850.290760] Lustre: 111170:0:(osd_compat.c:1353:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 2855.810886] Lustre: DEBUG MARKER: oleg635-server.virtnet: executing set_default_debug -1 all [ 2856.129738] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 2865.097081] Lustre: DEBUG MARKER: oleg635-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 2869.883306] Lustre: DEBUG MARKER: oleg635-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 [ 2870.240517] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [ 2875.482941] Lustre: DEBUG MARKER: oleg635-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid 50 [ 2875.799775] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [ 2881.505057] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 2881.520035] LustreError: Skipped 3 previous similar messages [ 2881.523403] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2881.552331] Lustre: Skipped 5 previous similar messages [ 2881.555835] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 2881.560224] Lustre: Skipped 13 previous similar messages [ 2883.586830] Lustre: server umount lustre-OST0000 complete [ 2901.983437] Lustre: lustre-MDT0000 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 2902.124345] Lustre: server umount lustre-MDT0000 complete [ 2905.305602] LustreError: 109195:0:(ldlm_lockd.c:2580:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1787193179 with bad export cookie 16876731856266337365 [ 2905.308139] LustreError: MGC192.168.206.135@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 2905.315540] LustreError: 109195:0:(ldlm_lockd.c:2580:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 2905.464875] Lustre: server umount lustre-MDT0001 complete [ 2914.585806] Lustre: DEBUG MARKER: == conf-sanity test 87: check if MDT inode can hold EAs with N stripes properly ========================================================== 22:33:06 (1787193186) [ 2923.674942] Lustre: DEBUG MARKER: oleg635-server.virtnet: executing unload_modules_local [ 2926.582343] Key type lgssc unregistered [ 2926.826367] LNet: 113659:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 2926.833501] LNetError: 113659:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 2926.849533] LNet: Removed LNI 192.168.206.135@tcp [ 2927.538816] Key type .llcrypt unregistered [ 2927.541908] Key type ._llcrypt unregistered [ 2943.960367] Key type ._llcrypt registered [ 2943.963968] Key type .llcrypt registered [ 2944.039823] Lustre: DEBUG MARKER: oleg635-server.virtnet: executing set_hostid [ 2959.067037] Lustre: DEBUG MARKER: oleg635-server.virtnet: executing load_modules_local [ 2959.748886] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 2959.819621] alg: No test for adler32 (adler32-zlib) [ 2960.879895] Lustre: Lustre: Build Version: 2.17.57_44_gafaaa93 [ 2961.109447] LNet: Added LNI 192.168.206.135@tcp [8/256/0/180] [ 2962.743124] Key type lgssc registered [ 2963.744250] Lustre: Echo OBD driver; http://www.lustre.org/ [ 2997.006456] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 2997.025199] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 2998.243988] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 2998.347423] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 2998.351761] Lustre: lustre-MDT0000: new disk, initializing [ 2998.430301] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 2998.450898] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 3002.371395] Lustre: DEBUG MARKER: oleg635-server.virtnet: executing set_default_debug -1 all [ 3012.328810] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 3012.509641] Lustre: lustre-OST0000: new disk, initializing [ 3012.516939] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 3012.521549] Lustre: Skipped 1 previous similar message [ 3012.525802] Lustre: 117830:0:(osd_compat.c:1353:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 3012.593021] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 3015.238670] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:0:ost [ 3015.247396] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:0:ost] [ 3015.273428] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x240000400 [ 3018.491515] Lustre: DEBUG MARKER: oleg635-server.virtnet: executing set_default_debug -1 all [ 3028.220900] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 3028.370170] Lustre: lustre-OST0001: new disk, initializing [ 3028.375167] Lustre: srv-lustre-OST0001: No data found on store. Initialize space. [ 3028.382822] Lustre: 118692:0:(osd_compat.c:1353:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 3028.473548] Lustre: lustre-OST0001: Imperative Recovery not enabled, recovery window 60-180 [ 3034.272168] Lustre: DEBUG MARKER: oleg635-server.virtnet: executing set_default_debug -1 all [ 3035.768314] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:1:ost [ 3035.781659] Lustre: cli-lustre-OST0001-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:1:ost] [ 3035.818116] Lustre: lustre-OST0001-osc-MDT0000: update sequence from 0x100010000 to 0x280000400 [ 3045.858147] Lustre: lustre-MDT0000-lwp-OST0000: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 3045.867786] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 3045.873137] Lustre: Skipped 1 previous similar message [ 3049.461192] Lustre: server umount lustre-MDT0000 complete [ 3056.703438] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: (null) [ 3060.943233] LustreError: 117018:0:(ldlm_lockd.c:2580:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1787193334 with bad export cookie 10554803317495160494 [ 3060.958229] LustreError: MGC192.168.206.135@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 3060.959810] LustreError: 117018:0:(ldlm_lockd.c:2580:ldlm_cancel_handler()) Skipped 2 previous similar messages [ 3061.061628] Lustre: server umount lustre-OST0000 complete [ 3064.303835] Lustre: server umount lustre-OST0001 complete [ 3071.241509] Lustre: DEBUG MARKER: == conf-sanity test 88a: tunefs.lustre persists a --mountfsoptions-only change ========================================================== 22:35:43 (1787193343) [ 3089.200541] Lustre: DEBUG MARKER: == conf-sanity test 88b: check the default mount options can be overridden ========================================================== 22:36:00 (1787193360) [ 3098.431971] Lustre: DEBUG MARKER: oleg635-server.virtnet: executing load_modules_local [ 3121.131512] Lustre: DEBUG MARKER: == conf-sanity test 89: check tunefs --param and --erase-param{s} options ========================================================== 22:36:32 (1787193392) [ 3158.044652] Lustre: DEBUG MARKER: oleg635-server.virtnet: executing set_hostid [ 3167.792425] Lustre: DEBUG MARKER: oleg635-server.virtnet: executing load_modules_local [ 3200.257703] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 3200.472350] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 3200.512497] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 3200.523173] Lustre: lustre-MDT0000: new disk, initializing [ 3200.580901] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 3200.592043] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 3204.118681] Lustre: DEBUG MARKER: oleg635-server.virtnet: executing set_default_debug -1 all [ 3213.080536] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 3213.168367] Lustre: 126449:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 3213.210903] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [ 3213.214519] Lustre: Skipped 1 previous similar message [ 3213.217684] Lustre: lustre-MDT0001: new disk, initializing [ 3213.283943] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 3213.320330] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [ 3213.330021] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 3217.349831] Lustre: DEBUG MARKER: oleg635-server.virtnet: executing set_default_debug -1 all [ 3228.298816] Lustre: DEBUG MARKER: oleg635-client.virtnet: executing wait_import_state_mount FULL failover.node=192.0.2.254@tcp0,mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 3234.081911] Lustre: DEBUG MARKER: oleg635-client.virtnet: executing wait_import_state_mount FULL failover.node=192.0.2.254@tcp0,mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 3241.710795] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 3241.960437] Lustre: lustre-OST0000: new disk, initializing [ 3241.964614] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 3241.970505] Lustre: 128417:0:(osd_compat.c:1353:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 3242.034626] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 3247.782735] Lustre: DEBUG MARKER: oleg635-server.virtnet: executing set_default_debug -1 all [ 3249.191835] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [ 3249.200546] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 3249.311170] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 3256.742934] Lustre: DEBUG MARKER: oleg635-client.virtnet: executing wait_import_state_mount (FULL|IDLE) failover.node=192.0.2.254@tcp0,osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 3260.857484] Lustre: DEBUG MARKER: oleg635-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 [ 3261.033566] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [ 3265.451913] Lustre: DEBUG MARKER: oleg635-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid 50 [ 3265.650339] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [ 3269.601855] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 3269.624807] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 3269.635199] Lustre: Skipped 1 previous similar message [ 3269.638449] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 3270.119834] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 3273.494510] Lustre: server umount lustre-OST0000 complete [ 3279.847899] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 3279.857975] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 3279.872904] Lustre: Skipped 1 previous similar message [ 3279.879783] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 3279.885177] Lustre: Skipped 2 previous similar messages [ 3282.665723] Lustre: server umount lustre-MDT0000 complete [ 3285.917486] LustreError: 126442:0:(ldlm_lockd.c:2580:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1787193559 with bad export cookie 10554803317495161719 [ 3285.919064] LustreError: MGC192.168.206.135@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 3285.926947] LustreError: 126442:0:(ldlm_lockd.c:2580:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 3286.078053] Lustre: server umount lustre-MDT0001 complete [ 3294.156439] Lustre: DEBUG MARKER: == conf-sanity test 90a: check max_mod_rpcs_in_flight is enforced ========================================================== 22:39:25 (1787193565) [ 3300.652639] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 3301.120812] LustreError: 130831:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 3301.140351] LustreError: 130831:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 3 previous similar messages [ 3301.202602] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 3305.868374] Lustre: DEBUG MARKER: oleg635-server.virtnet: executing set_default_debug -1 all [ 3306.467356] LustreError: 130832:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 3310.562873] LustreError: 130831:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 3315.379595] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 3321.397537] Lustre: DEBUG MARKER: oleg635-server.virtnet: executing set_default_debug -1 all [ 3330.465667] Lustre: DEBUG MARKER: oleg635-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 3337.920331] Lustre: DEBUG MARKER: oleg635-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 3345.235650] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 3345.901378] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 3345.912734] Lustre: Skipped 1 previous similar message [ 3352.245809] Lustre: DEBUG MARKER: oleg635-server.virtnet: executing set_default_debug -1 all [ 3359.994573] Lustre: DEBUG MARKER: oleg635-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 3374.881943] Lustre: *** cfs_fail_loc=159, val=0*** [ 3391.082937] Lustre: lustre-MDT0000: Client b0bc0536-3f94-4d58-a1d3-cdf61af14fea (at 192.168.206.35@tcp) reconnecting [ 3392.739560] Lustre: *** cfs_fail_loc=159, val=0*** [ 3392.742516] Lustre: Skipped 5 previous similar messages [ 3407.949782] Lustre: lustre-MDT0000: Client b0bc0536-3f94-4d58-a1d3-cdf61af14fea (at 192.168.206.35@tcp) reconnecting [ 3413.472985] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 3413.479951] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 3413.497895] Lustre: Skipped 1 previous similar message [ 3413.511200] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 3413.518201] Lustre: Skipped 2 previous similar messages [ 3414.415421] Lustre: server umount lustre-OST0000 complete [ 3418.080533] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 3418.096955] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 3418.134451] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 3426.784230] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 3426.791699] Lustre: Skipped 3 previous similar messages [ 3427.908454] Lustre: server umount lustre-MDT0000 complete [ 3431.489779] LustreError: 130813:0:(ldlm_lockd.c:2580:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1787193705 with bad export cookie 10554803317495162594 [ 3431.491346] LustreError: MGC192.168.206.135@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 3431.519640] LustreError: 130813:0:(ldlm_lockd.c:2580:ldlm_cancel_handler()) Skipped 2 previous similar messages [ 3431.783327] Lustre: server umount lustre-MDT0001 complete [ 3442.456568] Lustre: DEBUG MARKER: oleg635-server.virtnet: executing unload_modules_local [ 3445.722386] Key type lgssc unregistered [ 3445.977536] LNet: 134631:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 3445.997623] LNetError: 134631:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 3446.025882] LNet: Removed LNI 192.168.206.135@tcp [ 3447.056351] Key type .llcrypt unregistered [ 3447.058954] Key type ._llcrypt unregistered [ 3468.901339] Key type ._llcrypt registered [ 3468.904163] Key type .llcrypt registered [ 3469.010166] Lustre: DEBUG MARKER: oleg635-server.virtnet: executing load_modules_local [ 3469.622366] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 3469.755558] alg: No test for adler32 (adler32-zlib) [ 3470.835642] Lustre: Lustre: Build Version: 2.17.57_44_gafaaa93 [ 3471.085918] LNet: Added LNI 192.168.206.135@tcp [8/256/0/180] [ 3472.800609] Key type lgssc registered [ 3473.817694] Lustre: Echo OBD driver; http://www.lustre.org/ [ 3479.810697] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 3479.831848] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 3481.038232] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 3485.091986] Lustre: DEBUG MARKER: oleg635-server.virtnet: executing set_default_debug -1 all [ 3490.275613] LustreError: 136024:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 3490.293655] LustreError: 136024:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 2 previous similar messages [ 3492.156713] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 3492.376514] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 3496.482756] Lustre: DEBUG MARKER: oleg635-server.virtnet: executing set_default_debug -1 all [ 3503.495866] Lustre: DEBUG MARKER: oleg635-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 3508.822562] Lustre: DEBUG MARKER: oleg635-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 3515.665677] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 3516.051181] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 3521.808514] Lustre: DEBUG MARKER: oleg635-server.virtnet: executing set_default_debug -1 all [ 3525.304027] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:42 to 0x280000401:65) [ 3528.170679] Lustre: DEBUG MARKER: oleg635-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 3537.744313] Lustre: *** cfs_fail_loc=159, val=0*** [ 3553.379512] Lustre: lustre-MDT0000: Client 41a2d43e-e3fc-4b20-9712-6b90404fad7a (at 192.168.206.35@tcp) reconnecting [ 3555.658597] Lustre: *** cfs_fail_loc=159, val=0*** [ 3571.285471] Lustre: lustre-MDT0001: Client 41a2d43e-e3fc-4b20-9712-6b90404fad7a (at 192.168.206.35@tcp) reconnecting [ 3572.716107] Lustre: *** cfs_fail_loc=159, val=0*** [ 3572.717972] Lustre: Skipped 3 previous similar messages [ 3588.160438] Lustre: lustre-MDT0001: Client 41a2d43e-e3fc-4b20-9712-6b90404fad7a (at 192.168.206.35@tcp) reconnecting [ 3607.700806] Lustre: 139081:0:(mdt_handler.c:64:mdt_max_mod_rpcs_per_client_set()) max_mod_rpcs_per_client is deprecated, set mdt.*.max_mod_rpcs_in_flight parameter instead [ 3610.345640] Lustre: *** cfs_fail_loc=159, val=0*** [ 3610.349693] Lustre: Skipped 1 previous similar message [ 3626.076170] Lustre: lustre-MDT0000: Client 59c37d3a-453f-4a54-b3ef-9c6c25a33454 (at 192.168.206.35@tcp) reconnecting [ 3627.556520] Lustre: *** cfs_fail_loc=159, val=0*** [ 3627.559749] Lustre: Skipped 14 previous similar messages [ 3642.993190] Lustre: lustre-MDT0000: Client 59c37d3a-453f-4a54-b3ef-9c6c25a33454 (at 192.168.206.35@tcp) reconnecting [ 3644.470754] Lustre: 139370:0:(mdt_handler.c:64:mdt_max_mod_rpcs_per_client_set()) max_mod_rpcs_per_client is deprecated, set mdt.*.max_mod_rpcs_in_flight parameter instead [ 3651.066487] LustreError: lustre-OST0000-osc-MDT0001: operation ost_destroy to node 0@lo failed: rc = -107 [ 3651.068031] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 3651.078668] LustreError: Skipped 2 previous similar messages [ 3651.102579] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 3653.606291] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 3653.620615] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 3653.634542] LustreError: Skipped 3 previous similar messages [ 3653.642791] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 3655.744859] Lustre: server umount lustre-OST0000 complete [ 3661.283165] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 3661.293568] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 3661.313262] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 3661.322707] Lustre: Skipped 1 previous similar message [ 3663.840632] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 3663.843238] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 3666.102440] Lustre: server umount lustre-MDT0000 complete [ 3668.967901] LustreError: 139140:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 3668.989613] LustreError: 139140:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 1 previous similar message [ 3669.679599] LustreError: 136004:0:(ldlm_lockd.c:2580:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1787193943 with bad export cookie 10911548913725311246 [ 3669.686951] LustreError: MGC192.168.206.135@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 3669.705125] LustreError: 136004:0:(ldlm_lockd.c:2580:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 3669.958525] Lustre: server umount lustre-MDT0001 complete [ 3680.999396] Lustre: DEBUG MARKER: oleg635-server.virtnet: executing unload_modules_local [ 3683.434796] Key type lgssc unregistered [ 3683.707810] LNet: 140426:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 3683.724648] LNetError: 140426:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 3683.737467] LNet: Removed LNI 192.168.206.135@tcp [ 3684.595185] Key type .llcrypt unregistered [ 3684.598659] Key type ._llcrypt unregistered [ 3706.971427] Key type ._llcrypt registered [ 3706.974331] Key type .llcrypt registered [ 3707.050674] Lustre: DEBUG MARKER: oleg635-server.virtnet: executing load_modules_local [ 3707.862957] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 3707.881386] alg: No test for adler32 (adler32-zlib) [ 3708.917052] Lustre: Lustre: Build Version: 2.17.57_44_gafaaa93 [ 3709.208812] LNet: Added LNI 192.168.206.135@tcp [8/256/0/180] [ 3710.888134] Key type lgssc registered [ 3711.892342] Lustre: Echo OBD driver; http://www.lustre.org/ [ 3719.046873] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 3719.067501] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 3720.396077] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 3724.244759] Lustre: DEBUG MARKER: oleg635-server.virtnet: executing set_default_debug -1 all [ 3730.917322] LustreError: 141845:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 3730.944088] LustreError: 141845:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 2 previous similar messages [ 3732.653960] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 3732.884910] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 3737.079411] Lustre: DEBUG MARKER: oleg635-server.virtnet: executing set_default_debug -1 all [ 3743.595859] Lustre: DEBUG MARKER: oleg635-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 3748.692941] Lustre: DEBUG MARKER: oleg635-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 3754.853836] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 3755.155430] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 3758.280717] Lustre: lustre-OST0000: new connection from lustre-MDT0001-mdtlov (cleaning up unused objects from 0x280000400:72 to 0x280000400:97) [ 3760.507293] Lustre: DEBUG MARKER: oleg635-server.virtnet: executing set_default_debug -1 all [ 3766.481276] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:85 to 0x280000401:129) [ 3766.745723] Lustre: DEBUG MARKER: oleg635-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 3771.536711] Lustre: 144423:0:(mdt_handler.c:64:mdt_max_mod_rpcs_per_client_set()) max_mod_rpcs_per_client is deprecated, set mdt.*.max_mod_rpcs_in_flight parameter instead [ 3774.017811] Lustre: 144473:0:(mdt_handler.c:64:mdt_max_mod_rpcs_per_client_set()) max_mod_rpcs_per_client is deprecated, set mdt.*.max_mod_rpcs_in_flight parameter instead [ 3775.425967] Lustre: Modifying parameter lustre.mdc.lustre-MDT0000-mdc-*.max_rpcs_in_flight=8 in log params [ 3776.195776] Lustre: Setting parameter lustre.mdc.lustre-MDT0000-mdc-*.max_mod_rpcs_in_flight=8 in log params [ 3779.466976] Lustre: 144618:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre/mdc.lustre-MDT0000-mdc-*.max_rpcs_in_flight=8 (mode = 0) failed: rc = -17 [ 3780.182917] Lustre: Modifying parameter lustre.mdc.lustre-MDT0000-mdc-*.max_mod_rpcs_in_flight=7 in log params [ 3781.038727] Lustre: 144717:0:(mdt_handler.c:64:mdt_max_mod_rpcs_per_client_set()) max_mod_rpcs_per_client is deprecated, set mdt.*.max_mod_rpcs_in_flight parameter instead [ 3784.160225] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 3784.165079] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 3784.172177] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 3785.186353] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 3785.188044] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 3785.201085] Lustre: Skipped 1 previous similar message [ 3790.283281] Lustre: server umount lustre-OST0000 complete [ 3794.400178] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 3794.407464] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 3794.415386] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 3799.666596] Lustre: server umount lustre-MDT0000 complete [ 3800.546925] LustreError: 141844:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 3800.567986] LustreError: 141844:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 1 previous similar message [ 3803.613667] LustreError: 141826:0:(ldlm_lockd.c:2580:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1787194077 with bad export cookie 4858465562152033958 [ 3803.627097] LustreError: MGC192.168.206.135@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 3803.636536] LustreError: 141826:0:(ldlm_lockd.c:2580:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 3803.866756] Lustre: server umount lustre-MDT0001 complete [ 3815.438971] Lustre: DEBUG MARKER: oleg635-server.virtnet: executing unload_modules_local [ 3819.043566] Key type lgssc unregistered [ 3819.430455] LNet: 145779:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 3819.449093] LNetError: 145779:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 3819.474847] LNet: Removed LNI 192.168.206.135@tcp [ 3820.870290] Key type .llcrypt unregistered [ 3820.872430] Key type ._llcrypt unregistered [ 3843.253717] Key type ._llcrypt registered [ 3843.255727] Key type .llcrypt registered [ 3843.349661] Lustre: DEBUG MARKER: oleg635-server.virtnet: executing load_modules_local [ 3844.403236] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 3844.507725] alg: No test for adler32 (adler32-zlib) [ 3845.520730] Lustre: Lustre: Build Version: 2.17.57_44_gafaaa93 [ 3845.721065] LNet: Added LNI 192.168.206.135@tcp [8/256/0/180] [ 3847.367382] Key type lgssc registered [ 3848.331726] Lustre: Echo OBD driver; http://www.lustre.org/ [ 3853.947925] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 3853.965625] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 3855.321448] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 3859.138978] Lustre: DEBUG MARKER: oleg635-server.virtnet: executing set_default_debug -1 all [ 3865.570578] LustreError: 147188:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 3865.589179] LustreError: 147188:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 2 previous similar messages [ 3865.965540] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 3866.158119] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 3869.828985] Lustre: DEBUG MARKER: oleg635-server.virtnet: executing set_default_debug -1 all [ 3877.439971] Lustre: DEBUG MARKER: oleg635-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 3885.050746] Lustre: DEBUG MARKER: oleg635-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 3893.604208] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 3893.919601] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 3900.925098] Lustre: DEBUG MARKER: oleg635-server.virtnet: executing set_default_debug -1 all [ 3901.131476] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:85 to 0x280000401:161) [ 3907.767903] Lustre: DEBUG MARKER: oleg635-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 3910.334699] Lustre: lustre-OST0000: new connection from lustre-MDT0001-mdtlov (cleaning up unused objects from 0x280000400:72 to 0x280000400:129) [ 3913.877485] Lustre: *** cfs_fail_loc=159, val=0*** [ 3930.233236] Lustre: lustre-MDT0000: Client 647adaa2-1646-4cbe-a390-6ee1b49d30b9 (at 192.168.206.35@tcp) reconnecting [ 3935.747196] LustreError: lustre-OST0000-osc-MDT0000: operation ost_destroy to node 0@lo failed: rc = -107 [ 3935.759455] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 3935.772479] LustreError: Skipped 1 previous similar message [ 3935.812543] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 3936.250250] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 3936.262233] LustreError: Skipped 5 previous similar messages [ 3936.269542] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 3936.760782] Lustre: server umount lustre-OST0000 complete [ 3941.347849] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 3941.352524] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 3941.354254] Lustre: Skipped 1 previous similar message [ 3941.377826] Lustre: Skipped 3 previous similar messages [ 3946.366514] Lustre: server umount lustre-MDT0000 complete [ 3946.467136] LustreError: 147188:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 3946.499079] LustreError: 147188:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 1 previous similar message [ 3949.583540] LustreError: 147169:0:(ldlm_lockd.c:2580:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1787194223 with bad export cookie 2099810516555609289 [ 3949.585655] LustreError: MGC192.168.206.135@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 3949.594807] LustreError: 147169:0:(ldlm_lockd.c:2580:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 3949.760435] Lustre: server umount lustre-MDT0001 complete [ 3958.673678] Lustre: DEBUG MARKER: oleg635-server.virtnet: executing unload_modules_local [ 3961.333559] Key type lgssc unregistered [ 3961.650781] LNet: 150854:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 3961.665353] LNetError: 150854:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 3962.729859] LNet: Removed LNI 192.168.206.135@tcp [ 3963.510347] Key type .llcrypt unregistered [ 3963.512939] Key type ._llcrypt unregistered [ 3984.925294] Key type ._llcrypt registered [ 3984.932241] Key type .llcrypt registered [ 3985.042259] Lustre: DEBUG MARKER: oleg635-server.virtnet: executing load_modules_local [ 3985.858135] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 3985.930254] alg: No test for adler32 (adler32-zlib) [ 3987.000343] Lustre: Lustre: Build Version: 2.17.57_44_gafaaa93 [ 3987.247688] LNet: Added LNI 192.168.206.135@tcp [8/256/0/180] [ 3988.935828] Key type lgssc registered [ 3989.900642] Lustre: Echo OBD driver; http://www.lustre.org/ [ 3996.077636] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 3996.099985] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 3997.365658] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 4001.005517] Lustre: DEBUG MARKER: oleg635-server.virtnet: executing set_default_debug -1 all [ 4007.906169] LustreError: 152268:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 4007.929276] LustreError: 152268:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 2 previous similar messages [ 4008.028603] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 4008.278975] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 4012.029918] Lustre: DEBUG MARKER: oleg635-server.virtnet: executing set_default_debug -1 all [ 4019.169596] Lustre: DEBUG MARKER: oleg635-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 4024.194701] Lustre: DEBUG MARKER: oleg635-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 4029.320652] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 4029.626653] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 4032.693169] Lustre: lustre-OST0000: new connection from lustre-MDT0001-mdtlov (cleaning up unused objects from 0x280000400:72 to 0x280000400:161) [ 4034.021584] Lustre: DEBUG MARKER: oleg635-server.virtnet: executing set_default_debug -1 all [ 4039.796379] Lustre: DEBUG MARKER: oleg635-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 4042.927694] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:170 to 0x280000401:193) [ 4062.174097] Lustre: lustre-OST0000: evicting 223519c9-4879-4305-b45f-f229d39f9a6a (at 192.168.206.35@tcp) by administrative request [ 4062.180896] Lustre: Skipped 1 previous similar message [ 4089.314298] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4089.335416] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 4089.335827] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 4089.354215] Lustre: Skipped 1 previous similar message [ 4094.437779] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 4094.443514] Lustre: Skipped 1 previous similar message [ 4095.278698] Lustre: server umount lustre-OST0000 complete [ 4099.554900] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4099.557277] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 4099.568544] Lustre: Skipped 2 previous similar messages [ 4104.672530] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 4104.677026] Lustre: Skipped 2 previous similar messages [ 4109.797745] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 4109.808496] Lustre: Skipped 1 previous similar message [ 4112.351123] Lustre: lustre-MDT0000 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 4112.456095] Lustre: server umount lustre-MDT0000 complete [ 4114.914961] LustreError: 152269:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 4114.941239] LustreError: 152269:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 1 previous similar message [ 4115.258767] LustreError: 152248:0:(ldlm_lockd.c:2580:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1787194388 with bad export cookie 9643989312624616886 [ 4115.263234] LustreError: MGC192.168.206.135@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 4115.277119] LustreError: 152248:0:(ldlm_lockd.c:2580:ldlm_cancel_handler()) Skipped 3 previous similar messages [ 4115.467808] Lustre: server umount lustre-MDT0001 complete [ 4124.458618] Lustre: DEBUG MARKER: oleg635-server.virtnet: executing unload_modules_local [ 4126.743573] Key type lgssc unregistered [ 4127.015373] LNet: 156343:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 4127.043442] LNetError: 156343:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 4127.075298] LNet: Removed LNI 192.168.206.135@tcp [ 4127.739296] Key type .llcrypt unregistered [ 4127.741306] Key type ._llcrypt unregistered [ 4186.377595] Key type ._llcrypt registered [ 4186.382698] Key type .llcrypt registered [ 4186.454429] Lustre: DEBUG MARKER: oleg635-server.virtnet: executing load_modules_local [ 4187.171683] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 4187.232218] alg: No test for adler32 (adler32-zlib) [ 4188.204327] Lustre: Lustre: Build Version: 2.17.57_44_gafaaa93 [ 4188.382352] LNet: Added LNI 192.168.206.135@tcp [8/256/0/180] [ 4190.015157] Key type lgssc registered [ 4190.859422] Lustre: Echo OBD driver; http://www.lustre.org/ [ 4196.239644] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 4196.269037] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 4197.528568] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 4201.173628] Lustre: DEBUG MARKER: oleg635-server.virtnet: executing set_default_debug -1 all [ 4207.460958] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 4207.674866] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 4210.763144] Lustre: DEBUG MARKER: oleg635-server.virtnet: executing set_default_debug -1 all [ 4216.164532] Lustre: DEBUG MARKER: oleg635-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 4220.578421] Lustre: DEBUG MARKER: oleg635-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 4226.049417] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 4226.350026] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 4231.034898] Lustre: DEBUG MARKER: oleg635-server.virtnet: executing set_default_debug -1 all [ 4232.510171] Lustre: lustre-OST0000: new connection from lustre-MDT0001-mdtlov (cleaning up unused objects from 0x280000400:72 to 0x280000400:193) [ 4237.539554] Lustre: DEBUG MARKER: oleg635-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 4243.762509] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:195 to 0x280000401:225) [ 4258.273772] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 4258.281807] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4258.304024] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 4259.300548] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 4259.302697] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 4259.308450] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4261.595238] Lustre: server umount lustre-OST0000 complete [ 4269.035393] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 4269.053464] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4269.070887] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 4269.077181] Lustre: Skipped 1 previous similar message [ 4274.656667] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 4274.666190] Lustre: Skipped 2 previous similar messages [ 4279.263536] Lustre: lustre-MDT0000 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 4279.396976] Lustre: server umount lustre-MDT0000 complete [ 4279.782664] LustreError: 158809:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 4279.806800] LustreError: 158809:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 3 previous similar messages [ 4282.579877] LustreError: 158787:0:(ldlm_lockd.c:2580:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1787194556 with bad export cookie 16405173147393896623 [ 4282.593312] LustreError: 158787:0:(ldlm_lockd.c:2580:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 4282.597203] LustreError: MGC192.168.206.135@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 4282.733848] Lustre: server umount lustre-MDT0001 complete [ 4291.966617] Lustre: DEBUG MARKER: oleg635-server.virtnet: executing unload_modules_local [ 4294.318632] Key type lgssc unregistered [ 4294.626794] LNet: 162374:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 4294.635823] LNetError: 162374:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 4294.657776] LNet: Removed LNI 192.168.206.135@tcp [ 4295.460339] Key type .llcrypt unregistered [ 4295.464250] Key type ._llcrypt unregistered [ 4317.615491] Key type ._llcrypt registered [ 4317.619040] Key type .llcrypt registered [ 4317.696832] Lustre: DEBUG MARKER: oleg635-server.virtnet: executing load_modules_local [ 4318.706452] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 4318.828053] alg: No test for adler32 (adler32-zlib) [ 4319.865548] Lustre: Lustre: Build Version: 2.17.57_44_gafaaa93 [ 4320.037608] LNet: Added LNI 192.168.206.135@tcp [8/256/0/180] [ 4321.696602] Key type lgssc registered [ 4322.574794] Lustre: Echo OBD driver; http://www.lustre.org/ [ 4333.470265] Lustre: DEBUG MARKER: == conf-sanity test 100: check lshowmount lists MGS, MDT, OST and 0@lo ========================================================== 22:56:45 (1787194605) [ 4344.760315] Lustre: DEBUG MARKER: oleg635-server.virtnet: executing set_hostid [ 4352.321105] Lustre: DEBUG MARKER: oleg635-server.virtnet: executing load_modules_local [ 4384.068538] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 4384.085344] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 4385.271580] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 4385.306692] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 4385.316577] Lustre: lustre-MDT0000: new disk, initializing [ 4385.403705] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 4385.421489] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 4390.327511] Lustre: DEBUG MARKER: oleg635-server.virtnet: executing set_default_debug -1 all [ 4399.483209] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 4399.584342] Lustre: 166638:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 4399.646452] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [ 4399.655807] Lustre: Skipped 1 previous similar message [ 4399.672210] Lustre: lustre-MDT0001: new disk, initializing [ 4399.791860] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 4399.818564] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [ 4399.831712] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 4403.893565] Lustre: DEBUG MARKER: oleg635-server.virtnet: executing set_default_debug -1 all [ 4415.065350] Lustre: DEBUG MARKER: oleg635-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 4422.237974] Lustre: DEBUG MARKER: oleg635-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 4429.701524] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 4430.017320] Lustre: lustre-OST0000: new disk, initializing [ 4430.023518] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 4430.035808] Lustre: 168607:0:(osd_compat.c:1353:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 4430.127537] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 4435.319525] Lustre: DEBUG MARKER: oleg635-server.virtnet: executing set_default_debug -1 all [ 4437.067876] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [ 4437.076536] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 4437.163804] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 4444.394084] Lustre: DEBUG MARKER: oleg635-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 4452.832617] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 4452.834053] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4452.839811] LustreError: Skipped 1 previous similar message [ 4452.840441] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 4452.847042] Lustre: Skipped 1 previous similar message [ 4452.854846] Lustre: Skipped 1 previous similar message [ 4453.878396] Lustre: server umount lustre-OST0000 complete [ 4456.416914] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 4456.423524] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4456.438533] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 4457.953815] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 4457.954784] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4462.417058] Lustre: server umount lustre-MDT0000 complete [ 4465.185665] LustreError: 166629:0:(ldlm_lockd.c:2580:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1787194738 with bad export cookie 9748580723739893461 [ 4465.188151] LustreError: MGC192.168.206.135@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 4465.191779] LustreError: 166629:0:(ldlm_lockd.c:2580:ldlm_cancel_handler()) Skipped 3 previous similar messages [ 4465.313536] Lustre: server umount lustre-MDT0001 complete [ 4472.869752] Lustre: DEBUG MARKER: oleg635-server.virtnet: executing unload_modules_local [ 4474.571416] Key type lgssc unregistered [ 4474.821085] LNet: 170511:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 4474.829370] LNetError: 170511:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 4474.846133] LNet: Removed LNI 192.168.206.135@tcp [ 4475.408740] Key type .llcrypt unregistered [ 4475.412094] Key type ._llcrypt unregistered [ 4495.288690] Key type ._llcrypt registered [ 4495.290560] Key type .llcrypt registered [ 4495.414291] Lustre: DEBUG MARKER: oleg635-server.virtnet: executing load_modules_local [ 4496.912635] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 4496.945970] alg: No test for adler32 (adler32-zlib) [ 4497.995837] Lustre: Lustre: Build Version: 2.17.57_44_gafaaa93 [ 4498.239845] LNet: Added LNI 192.168.206.135@tcp [8/256/0/180] [ 4499.904085] Key type lgssc registered [ 4501.053622] Lustre: Echo OBD driver; http://www.lustre.org/ [ 4507.159141] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 4507.179536] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 4508.480257] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 4511.758627] Lustre: DEBUG MARKER: oleg635-server.virtnet: executing set_default_debug -1 all [ 4518.240111] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 4518.447972] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 4522.411798] Lustre: DEBUG MARKER: oleg635-server.virtnet: executing set_default_debug -1 all [ 4529.427294] Lustre: DEBUG MARKER: oleg635-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 4534.876922] Lustre: DEBUG MARKER: oleg635-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 4541.138891] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 4541.415540] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 4547.544977] Lustre: DEBUG MARKER: oleg635-server.virtnet: executing set_default_debug -1 all [ 4553.630279] Lustre: DEBUG MARKER: oleg635-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 4554.734854] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:3 to 0x280000401:33) [ 4563.109153] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 4563.130813] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4563.146133] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 4563.151639] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4563.158586] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 4563.902550] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 4563.922668] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4563.933709] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 4563.939066] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4563.953478] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 4564.711763] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4565.435395] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 4565.438550] Lustre: Skipped 1 previous similar message [ 4565.451640] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4565.468882] Lustre: Skipped 1 previous similar message [ 4565.480056] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 4565.487288] Lustre: Skipped 1 previous similar message [ 4565.498545] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4565.511402] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 4565.523957] Lustre: Skipped 1 previous similar message [ 4566.328744] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4567.016578] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4567.751915] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 4567.756505] Lustre: Skipped 2 previous similar messages [ 4567.774550] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4567.785432] Lustre: Skipped 2 previous similar messages [ 4567.791881] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 4567.800949] Lustre: Skipped 2 previous similar messages [ 4567.803871] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4567.811332] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 4567.815766] Lustre: Skipped 2 previous similar messages [ 4568.615207] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4569.387160] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4570.195648] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4571.144162] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4571.833592] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 4571.839967] Lustre: Skipped 4 previous similar messages [ 4571.852722] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4571.859432] Lustre: Skipped 4 previous similar messages [ 4571.863229] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 4571.869371] Lustre: Skipped 4 previous similar messages [ 4571.872912] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4571.882312] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 4571.888415] Lustre: Skipped 4 previous similar messages [ 4572.577134] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4573.378453] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4574.347422] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4575.146294] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4575.865874] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4576.519832] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4577.315730] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4578.082906] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4578.898168] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4579.731236] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4580.595544] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 4580.604813] Lustre: Skipped 10 previous similar messages [ 4580.628986] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4580.643510] Lustre: Skipped 10 previous similar messages [ 4580.654434] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 4580.659706] Lustre: Skipped 10 previous similar messages [ 4580.668711] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4580.680436] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 4580.684632] Lustre: Skipped 10 previous similar messages [ 4581.542785] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4582.323792] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4583.007376] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4583.931833] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4584.775521] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4585.667587] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4586.381713] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4587.227804] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4588.105394] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4589.136332] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4589.853473] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4590.754194] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4591.549363] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4592.370718] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4593.268763] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4594.358858] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4595.063413] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4595.793518] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4596.408447] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4597.078671] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 4597.088791] Lustre: Skipped 19 previous similar messages [ 4597.102770] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4597.111875] Lustre: Skipped 19 previous similar messages [ 4597.115087] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 4597.125356] Lustre: Skipped 19 previous similar messages [ 4597.138191] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4597.156187] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 4597.159424] Lustre: Skipped 19 previous similar messages [ 4598.030391] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4598.854823] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4599.752869] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4600.631984] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4601.425166] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4602.333383] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4603.059072] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4603.938558] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4604.888275] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4605.888775] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4606.766941] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4607.670684] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4608.245442] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4608.916859] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4609.718951] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4610.516968] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4611.217374] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4612.146089] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4612.865952] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4613.587442] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4614.243313] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4615.324879] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4615.935690] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4616.684824] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4617.540065] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4618.356751] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4619.093585] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4619.962949] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4621.068450] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4621.966841] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4622.766509] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4623.431151] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4624.287599] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4625.041660] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4625.842970] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4626.847051] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4628.020631] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4628.965463] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4629.727873] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 4629.732633] Lustre: Skipped 38 previous similar messages [ 4629.750642] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4629.767802] Lustre: Skipped 38 previous similar messages [ 4629.772803] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 4629.778429] Lustre: Skipped 38 previous similar messages [ 4629.781633] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4629.798188] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 4629.802294] Lustre: Skipped 38 previous similar messages [ 4630.722096] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4631.480538] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4632.249175] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4632.972135] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4633.646751] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4634.299986] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4634.953135] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4635.609601] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4636.243650] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4636.756221] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4637.274498] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4637.802440] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4638.416474] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4638.971585] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4639.498225] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4639.982312] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4640.670350] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4641.286372] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4641.950554] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4642.510397] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4643.174190] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4643.963643] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4644.566870] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4645.148201] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4645.789887] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4646.419188] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4647.263675] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4647.937738] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4648.669677] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4649.349945] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4650.183361] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4651.014777] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4651.688143] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4652.359253] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4652.995940] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4653.695106] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4654.288737] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4655.018666] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4655.955470] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4656.564727] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4657.271585] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4657.902315] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4658.676401] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4659.586855] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4660.551238] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4661.526294] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4662.165127] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4663.022154] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4663.888630] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4664.583533] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4665.541764] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4666.406322] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4667.279606] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4668.044052] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4668.737315] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4669.439411] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4670.295402] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4671.125109] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4671.952652] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4672.783295] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4673.450676] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4674.237423] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4675.040872] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4675.754076] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4676.515906] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4677.254312] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4677.936532] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4678.550340] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4679.327049] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4679.977640] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4680.572215] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4681.172734] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4681.840679] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4682.509496] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4683.135675] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4683.766348] LustreError: 171945:0:(osp_precreate.c:655:osp_precreate_send()) lustre-OST0000-osc-MDT0000: can't precreate: rc = -5 [ 4683.778524] LustreError: 171945:0:(osp_precreate.c:1359:osp_precreate_thread()) lustre-OST0000-osc-MDT0000: cannot precreate objects: rc = -5 [ 4683.790220] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4684.489728] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4685.174276] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4685.861808] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4686.579751] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4687.200077] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4687.694346] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4688.310933] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4688.965758] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4689.576950] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4690.311486] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4690.928670] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4691.731452] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4692.479220] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4693.173567] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4693.831197] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 4693.838748] Lustre: Skipped 90 previous similar messages [ 4693.849104] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4693.854827] Lustre: Skipped 90 previous similar messages [ 4693.859965] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 4693.864012] Lustre: Skipped 90 previous similar messages [ 4693.869982] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4693.882439] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 4693.885481] Lustre: Skipped 90 previous similar messages [ 4694.564543] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4695.205836] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4695.936880] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4696.571877] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4697.316302] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4698.009800] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4698.785514] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4699.584129] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4700.218620] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4700.995218] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4701.705918] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4702.395809] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4703.027876] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4703.701936] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4704.419340] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4705.245645] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4705.999096] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4706.734161] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4707.370715] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4707.934249] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4708.519773] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4709.092458] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4709.765872] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4710.370815] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4710.951795] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4711.513607] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4712.173212] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4712.940086] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4713.624228] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4714.304523] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4715.013653] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4715.707463] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4716.433699] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4717.141647] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4717.876350] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4718.552945] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4719.476899] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4720.302425] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4721.244988] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4722.063205] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4722.737801] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4723.485356] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4724.233613] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4724.997423] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4725.704875] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4726.415457] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4727.223151] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4728.074993] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4728.702736] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4729.419343] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4730.153769] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4730.943452] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4731.726075] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4732.636169] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4733.432715] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4734.262614] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4735.037628] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4736.238707] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4737.005940] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4737.812214] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4738.636551] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4739.647492] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4740.520048] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4741.209807] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4742.073326] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4743.036629] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4743.963664] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4744.961477] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4745.935946] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4747.044873] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4748.273719] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4749.156706] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4750.115111] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4751.248931] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4752.094241] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4752.947434] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4753.750567] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4754.599895] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4755.253946] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4755.961379] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4756.606624] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4757.504875] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4758.242142] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4758.977589] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4759.623750] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4760.403200] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4761.256577] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4762.202391] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4763.083382] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4763.966796] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4764.705371] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4765.625293] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4766.398432] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4767.376269] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4768.306503] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4769.504524] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4770.621614] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4771.606643] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4772.560440] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4773.439072] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4774.343239] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4775.433425] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4776.488572] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4777.798467] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4778.880365] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4779.872624] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4780.762949] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4781.666863] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4782.585946] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4783.367563] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4784.172576] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4785.014780] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4785.975373] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4786.633954] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4787.422575] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4788.132803] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4788.943684] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4789.818045] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4790.631219] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4791.393242] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4792.306931] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4793.016404] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4793.866189] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4794.728834] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4795.513374] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4796.277567] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4797.059707] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4797.816455] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4798.639981] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4799.443623] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4800.246064] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4800.958589] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4801.698409] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4802.330083] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4803.085036] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4803.710836] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4804.495666] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4805.117654] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4805.653891] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4806.427417] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4807.156483] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4807.922066] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4808.718904] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4809.503498] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4810.323231] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4811.087327] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4811.912424] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4812.533995] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4813.198265] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4813.829404] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4814.509867] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4815.178710] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4816.001197] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4816.763792] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4817.486333] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4818.200040] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4818.954676] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4819.801110] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4820.563287] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4821.457682] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4822.339911] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 4822.343919] Lustre: Skipped 160 previous similar messages [ 4822.372179] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4822.388452] Lustre: Skipped 160 previous similar messages [ 4822.396277] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 4822.402960] Lustre: Skipped 160 previous similar messages [ 4822.409506] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4822.427642] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 4822.437067] Lustre: Skipped 160 previous similar messages [ 4823.239429] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4824.081501] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4824.846695] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4825.541234] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4826.313122] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4827.032131] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4827.882210] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4828.645906] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4829.435728] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4830.050055] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4830.765529] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4831.398354] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4832.191419] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4832.887569] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4833.661124] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4834.298890] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4835.100738] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4835.839847] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4836.567893] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4837.239654] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4837.920074] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4838.785472] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4839.545338] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4840.278523] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4841.102923] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4842.006118] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4842.904946] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4843.664743] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4844.598269] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4845.404050] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4846.234290] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4847.201343] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4848.300423] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4849.533252] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4850.378679] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4851.295593] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4852.029269] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4853.029234] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4854.021699] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4855.100449] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4855.915246] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4857.083752] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4858.282796] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4859.293681] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4860.119511] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4860.965799] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4862.307666] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4863.268417] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4864.243716] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4864.995165] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4865.885103] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4867.151646] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4868.166212] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4869.258658] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4870.164066] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4871.112906] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4872.292068] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4873.435687] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4874.210356] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4875.075848] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4875.973232] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4876.944524] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4877.799609] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4878.558973] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4879.354253] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4880.182705] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4881.042798] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4881.807068] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4882.588709] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4883.548503] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4884.223996] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4885.132109] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4885.860737] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4886.747791] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4887.453289] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4888.151661] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4888.792284] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4889.503493] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4890.129260] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4890.773817] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4891.494339] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4892.256441] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4892.862514] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4893.665555] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4894.411978] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4895.181102] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4895.857511] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4896.658241] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4897.424281] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4898.174501] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4898.850800] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4899.496956] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4900.094202] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4900.803751] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4901.511795] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4902.205447] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4902.919937] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4903.683813] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4904.347213] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4905.109301] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4905.805597] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4906.565354] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4907.187824] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4907.904457] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4908.642283] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4909.432159] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4910.185790] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4911.048750] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4911.957275] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4912.827435] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4913.572259] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4914.329363] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4915.081392] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4915.868608] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4916.536371] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4917.242219] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4917.963180] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4918.706836] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4919.533783] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4920.214821] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4920.903823] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4921.574247] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4922.356105] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4923.214571] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4924.056461] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4924.999209] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4925.751844] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4926.593346] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4927.324260] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4928.174736] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4928.966646] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4929.897327] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4930.635814] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4931.520161] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4932.428147] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4933.219813] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4934.023481] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4934.933266] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4935.644225] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4936.477850] LustreError: 171945:0:(osp_precreate.c:655:osp_precreate_send()) lustre-OST0000-osc-MDT0000: can't precreate: rc = -5 [ 4936.488981] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4936.491585] LustreError: 171945:0:(osp_precreate.c:1359:osp_precreate_thread()) lustre-OST0000-osc-MDT0000: cannot precreate objects: rc = -5 [ 4937.261554] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4938.100508] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4938.962382] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4939.881382] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4940.656441] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4941.595425] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4942.453189] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4943.320352] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4944.091271] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4944.887276] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4945.693814] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4946.490802] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4947.128492] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4947.944637] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4948.761633] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4949.684984] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4950.456573] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4951.256862] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4952.022606] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4952.733873] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4953.586655] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4954.372827] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4955.040222] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4955.936217] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4956.711472] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4957.563371] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4958.474566] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4959.434422] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4960.389936] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4961.239721] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4962.221257] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4962.976305] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4963.839474] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4964.644736] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4965.451515] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4966.389875] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4967.290939] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4968.064715] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4968.924350] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4969.818466] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4970.865939] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4971.726574] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4972.596918] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4973.549924] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4974.565705] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4975.753681] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4976.688447] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4977.571251] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4978.269260] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4979.055258] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4979.792405] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4980.758219] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4981.578905] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4982.671611] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4983.826132] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4984.664432] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4985.767371] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4986.629139] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4987.653612] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4988.484570] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4989.641801] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4990.746661] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4991.550803] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4992.512128] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4993.196622] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4994.168378] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4994.962123] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4995.749925] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4996.543067] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4997.418090] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4998.097039] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4998.917876] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4999.790645] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5000.754581] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5001.501680] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5002.364138] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5003.130674] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5003.941369] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5004.640913] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5005.381530] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5006.252617] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5007.048324] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5007.880042] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5008.676905] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5009.542814] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5010.282233] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5011.093887] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5011.814251] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5012.546761] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5013.301177] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5014.166825] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5014.925802] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5015.709681] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5016.440172] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5017.170621] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5017.804790] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5018.485060] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5019.155209] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5019.869915] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5020.540761] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5021.227304] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5021.878243] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5022.535482] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5023.355448] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5024.250524] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5024.942170] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5025.648618] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5026.333765] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5027.048778] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5027.804221] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5028.636273] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5029.329249] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5030.108983] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5030.887245] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5031.687766] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5032.378319] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5033.110237] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5033.812785] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5034.472510] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5035.150945] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5035.842408] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5036.547039] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5037.307754] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5037.966526] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5038.586834] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5039.165562] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5039.808699] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5040.418941] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5041.222477] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5041.972219] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5042.842465] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5043.654101] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5044.367168] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5045.135296] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5045.777192] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5046.630369] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5047.451756] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5048.256891] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5048.910147] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5049.700585] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5050.351563] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5051.020528] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5051.735450] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5052.376481] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5053.047227] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5053.701103] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5054.410051] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5055.064235] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5055.665489] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5056.386263] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5057.086600] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5057.787275] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5058.427624] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5059.141086] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5059.856850] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5060.535200] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5061.272405] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5062.015462] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5062.812540] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5063.469808] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5064.116911] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5064.720501] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5065.407017] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5066.029323] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5066.626790] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5067.237450] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5067.885322] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5068.504918] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5069.128181] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5069.726600] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5070.402084] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5071.012967] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5071.649820] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5072.315189] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5073.026320] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5073.626409] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5074.322898] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5075.033233] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5075.708059] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5076.380827] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5077.167958] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5078.102793] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5078.923820] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 5078.928609] Lustre: Skipped 322 previous similar messages [ 5078.944289] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5078.953015] Lustre: Skipped 322 previous similar messages [ 5078.955344] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 5078.960238] Lustre: Skipped 322 previous similar messages [ 5078.964395] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5078.976143] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 5078.984171] Lustre: Skipped 322 previous similar messages [ 5079.675251] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5080.351408] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5081.082535] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5081.880455] LustreError: 171945:0:(osp_precreate.c:655:osp_precreate_send()) lustre-OST0000-osc-MDT0000: can't precreate: rc = -5 [ 5081.894464] LustreError: 171945:0:(osp_precreate.c:1359:osp_precreate_thread()) lustre-OST0000-osc-MDT0000: cannot precreate objects: rc = -5 [ 5081.905945] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5082.659562] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5083.388335] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5084.094856] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5084.802378] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5085.482032] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5086.207157] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5086.915201] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5087.691482] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5088.476174] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5089.366840] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5090.208822] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5090.950017] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5091.666978] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5092.511487] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5093.322868] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5094.201427] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5095.075261] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5095.868185] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5096.871312] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5097.603346] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5098.338131] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5099.095302] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5099.931797] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5100.736907] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5101.414687] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5102.213412] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5102.908105] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5103.642516] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5104.424246] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5105.202550] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5105.790487] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5106.412714] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5107.313492] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5108.009595] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5108.915707] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5109.792315] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5110.746330] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5111.577814] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5112.554931] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5113.583319] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5114.466732] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5115.143837] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5115.940871] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5116.635850] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5117.514336] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5118.198764] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5119.085616] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5119.931460] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5121.059651] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5121.904318] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5122.762897] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5123.511835] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5124.358066] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5125.123964] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5125.959423] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5126.713730] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5127.548349] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5128.330225] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5129.165969] LustreError: 171945:0:(osp_precreate.c:655:osp_precreate_send()) lustre-OST0000-osc-MDT0000: can't precreate: rc = -5 [ 5129.174140] LustreError: 171945:0:(osp_precreate.c:1359:osp_precreate_thread()) lustre-OST0000-osc-MDT0000: cannot precreate objects: rc = -5 [ 5129.182733] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5130.014115] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5130.805633] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5131.695728] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5132.657773] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5133.603948] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5134.422073] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5135.212933] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5135.994672] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5136.719148] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5137.604784] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5138.427578] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5139.211789] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5140.052915] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5140.838512] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5141.716530] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5142.680975] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5143.688838] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5144.511827] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5145.528620] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5146.444442] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5147.248314] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5148.200974] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5148.933332] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5149.865488] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5150.770452] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5151.526309] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5152.398127] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5153.386448] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5154.344141] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5155.166758] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5156.089296] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5156.918169] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5157.894869] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5158.692567] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5159.728553] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5160.580876] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5161.488645] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5162.300852] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5163.040671] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5163.935280] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5164.917256] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5165.784729] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5166.517775] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5167.346135] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5168.189860] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5168.943800] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5169.790278] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5170.625150] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5171.323687] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5172.182627] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5173.118265] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5174.062759] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5175.191173] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5175.990123] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5176.852344] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5177.690441] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5178.723878] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5179.648481] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5180.530831] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5181.503908] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5182.447883] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5183.312169] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5184.409536] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5185.274816] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5186.208588] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5186.978344] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5187.984539] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5188.684982] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5189.505158] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5190.200674] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5191.077328] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5191.935663] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5192.770136] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5193.675227] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5194.497234] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5195.189557] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5195.966210] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5196.687357] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5197.511366] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5198.308107] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5199.304028] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5200.044585] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5200.836649] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5201.737933] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5202.532853] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5203.350193] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5204.322384] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5205.276961] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5206.347154] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5207.312400] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5208.304333] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5209.174860] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5210.193787] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5211.186287] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5212.074321] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5213.099312] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5213.949133] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5214.872343] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5215.833917] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5216.772538] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5217.659419] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5218.649432] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5219.636589] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5220.484383] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5221.396521] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5222.478828] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5223.289035] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5224.251269] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5225.202052] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5226.327711] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5227.141199] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5227.999928] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5229.095062] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5230.059285] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5231.204460] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5232.359498] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5233.337959] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5234.229959] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5235.335321] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5236.320891] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5237.211224] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5238.152210] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5239.066820] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5240.009749] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5241.306780] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5242.207632] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5243.264725] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5244.441323] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5245.402694] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5246.463942] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5247.656692] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5248.965851] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5249.931825] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5251.084338] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5251.998733] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5253.126837] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5254.144880] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5255.373102] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5256.353642] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5257.571972] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5258.791955] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5259.661798] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5260.842417] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5261.888781] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5262.861707] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5264.118050] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5265.123300] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5266.019405] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5266.873275] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5268.114581] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5269.206927] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5270.089105] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5271.146514] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5272.295473] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5273.255924] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5274.261182] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5275.348668] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5276.442875] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5277.314685] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5278.107443] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5279.096923] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5280.115321] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5280.999912] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5282.153365] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5283.007727] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5283.879079] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5284.921630] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5285.740417] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5286.820415] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5287.507889] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5288.406278] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5289.417224] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5290.366880] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5291.054730] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5291.976590] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5292.876047] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5293.846231] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5294.718648] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5295.489751] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5296.401693] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5297.314384] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5298.179421] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5299.107706] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5299.939490] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5300.838218] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5301.594970] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5302.544557] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5303.325333] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5304.120130] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5304.824436] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5305.667063] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5306.503514] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5307.342119] LustreError: 171945:0:(osp_precreate.c:655:osp_precreate_send()) lustre-OST0000-osc-MDT0000: can't precreate: rc = -5 [ 5307.351129] LustreError: 171945:0:(osp_precreate.c:1359:osp_precreate_thread()) lustre-OST0000-osc-MDT0000: cannot precreate objects: rc = -5 [ 5307.357800] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5308.133483] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5308.990857] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5309.781251] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5310.724467] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5311.600346] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5312.408554] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5313.119374] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5314.109267] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5315.138849] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5316.029936] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5316.774666] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5317.468891] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5318.162576] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5318.928780] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5319.765085] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5320.514485] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5321.223392] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5321.965575] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5322.648830] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5323.512950] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5324.226571] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5325.063961] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5325.760936] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5326.512343] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5327.281653] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5328.032300] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5328.772500] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5329.530835] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5330.313308] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5330.983429] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5331.706986] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5332.409578] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5333.116537] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5333.831866] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5334.584923] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5335.566921] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5336.265707] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5337.071659] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5337.808886] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5338.587240] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5339.209848] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5339.996764] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5340.699252] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5341.441463] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5342.125778] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5342.880686] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5343.530558] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5344.383415] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5345.122736] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5345.770202] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5346.412334] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5347.034341] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5347.683900] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5348.350777] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5348.991216] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5349.694780] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5350.323658] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5351.058910] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5351.737653] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5352.399673] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5352.985866] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5353.753412] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5354.509383] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5355.279355] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5356.079283] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5356.833305] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5357.594581] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5358.349250] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5359.148612] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5360.002313] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5360.824335] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5361.540015] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5362.317092] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5363.049515] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5363.723661] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5364.470906] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5365.173176] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5365.885766] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5366.521465] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5367.202315] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5367.893480] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5368.656753] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5369.343758] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5370.221948] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5370.978483] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5371.777444] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5372.429137] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6420.457045] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 6420.485943] Lustre: Skipped 343 previous similar messages [ 6420.498584] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 6420.505093] Lustre: Skipped 1 previous similar message [ 6424.170036] Lustre: server umount lustre-OST0000 complete [ 6428.128047] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 6428.139406] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 6430.688534] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 6430.694283] Lustre: Skipped 1 previous similar message [ 6433.833613] Lustre: server umount lustre-MDT0000 complete [ 6435.808915] LustreError: 172651:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 6435.833481] LustreError: 172651:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 3 previous similar messages [ 6436.898233] LustreError: 171906:0:(ldlm_lockd.c:2580:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1787196710 with bad export cookie 5432799874302566763 [ 6436.902769] LustreError: MGC192.168.206.135@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 6436.915363] LustreError: 171906:0:(ldlm_lockd.c:2580:ldlm_cancel_handler()) Skipped 2 previous similar messages [ 6437.107569] Lustre: server umount lustre-MDT0001 complete [ 6446.035543] Lustre: DEBUG MARKER: oleg635-server.virtnet: executing unload_modules_local [ 6448.489933] Key type lgssc unregistered [ 6448.791605] LNet: 227170:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 6448.798661] LNetError: 227170:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 6448.819379] LNet: Removed LNI 192.168.206.135@tcp [ 6449.551148] Key type .llcrypt unregistered [ 6449.557136] Key type ._llcrypt unregistered [ 6474.648259] Key type ._llcrypt registered [ 6474.650739] Key type .llcrypt registered [ 6474.754809] Lustre: DEBUG MARKER: oleg635-server.virtnet: executing load_modules_local [ 6476.015960] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 6476.124254] alg: No test for adler32 (adler32-zlib) [ 6477.337971] Lustre: Lustre: Build Version: 2.17.57_44_gafaaa93 [ 6477.858417] LNet: Added LNI 192.168.206.135@tcp [8/256/0/180] [ 6479.503209] Key type lgssc registered [ 6480.909679] Lustre: Echo OBD driver; http://www.lustre.org/ [ 6488.265504] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 6488.275844] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 6489.606840] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 6494.837958] Lustre: DEBUG MARKER: oleg635-server.virtnet: executing set_default_debug -1 all [ 6499.811356] LustreError: 228587:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 6499.826703] LustreError: 228587:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 2 previous similar messages [ 6503.105086] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 6503.271537] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 6507.418373] Lustre: DEBUG MARKER: oleg635-server.virtnet: executing set_default_debug -1 all [ 6516.477560] Lustre: DEBUG MARKER: oleg635-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 6522.738439] Lustre: DEBUG MARKER: oleg635-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 6531.173340] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 6531.675906] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 6534.750873] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:49665 to 0x280000401:49696) [ 6539.583171] Lustre: DEBUG MARKER: oleg635-server.virtnet: executing set_default_debug -1 all [ 6547.614220] Lustre: DEBUG MARKER: oleg635-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 6553.550993] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 6555.577459] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 6555.585342] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 6555.593160] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6555.610081] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 6555.610900] LustreError: 228606:0:(osp_precreate.c:1268:osp_precreate_thread()) cfs_fail_timeout id 2107 sleeping for 20000ms [ 6558.179429] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 6558.207950] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 6558.234583] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 6559.873757] Lustre: server umount lustre-OST0000 complete [ 6575.703243] LustreError: 228606:0:(osp_precreate.c:1268:osp_precreate_thread()) cfs_fail_timeout id 2107 awake [ 6576.608638] Lustre: 227899:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1787196834/real 1787196834] req@00000000c2d67841 x1874011620428416/t0(0) o400->lustre-OST0000-osc-MDT0000@0@lo:28/4 lens 224/224 e 0 to 1 dl 1787196850 ref 1 fl Rpc:XNQr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 [ 6576.637985] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 6582.751285] Lustre: 227899:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1787196840/real 1787196840] req@00000000d485aba2 x1874011620429568/t0(0) o400->lustre-OST0000-osc-MDT0000@0@lo:28/4 lens 224/224 e 0 to 1 dl 1787196856 ref 1 fl Rpc:XNQr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 [ 6587.871120] Lustre: 227899:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1787196845/real 1787196845] req@000000009bf51005 x1874011620430592/t0(0) o400->lustre-OST0000-osc-MDT0000@0@lo:28/4 lens 224/224 e 0 to 1 dl 1787196861 ref 1 fl Rpc:XNQr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 [ 6591.456153] Lustre: 227898:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1787196849/real 1787196849] req@0000000042693609 x1874011620431488/t0(0) o400->lustre-OST0000-osc-MDT0000@0@lo:28/4 lens 224/224 e 0 to 1 dl 1787196865 ref 1 fl Rpc:XNQr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 [ 6594.848246] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 6595.129678] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 6596.172205] LustreError: lustre-OST0000-osc-MDT0001: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6596.209849] Lustre: lustre-OST0000-osc-MDT0001: Connection restored to 0@lo (at 0@lo) [ 6601.329645] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6601.349500] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 6602.493894] Lustre: DEBUG MARKER: oleg635-server.virtnet: executing set_default_debug -1 all [ 6611.302352] Lustre: DEBUG MARKER: oleg635-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 6613.057576] Lustre: DEBUG MARKER: osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid in FULL state after 0 sec [ 6618.560857] Lustre: DEBUG MARKER: oleg635-client.virtnet: executing wait_import_state (FULL|IDLE) osc.lustre-OST0000-osc-ffff9822c4c2c800.ost_server_uuid 50 [ 6619.914992] Lustre: DEBUG MARKER: osc.lustre-OST0000-osc-ffff9822c4c2c800.ost_server_uuid in FULL state after 0 sec [ 6626.216416] Lustre: server umount lustre-OST0000 complete [ 6631.393744] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 6631.409061] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 6631.435798] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 6637.536912] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 6637.554384] Lustre: Skipped 3 previous similar messages [ 6640.730426] Lustre: server umount lustre-MDT0000 complete [ 6642.667888] LustreError: 231432:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 6642.695940] LustreError: 231432:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 1 previous similar message [ 6643.684541] Lustre: 227900:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1787196901/real 1787196901] req@000000008d455354 x1874011620450560/t0(0) o13->lustre-OST0000-osc-MDT0001@0@lo:7/4 lens 224/368 e 0 to 1 dl 1787196917 ref 1 fl Rpc:XQr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 [ 6643.725245] Lustre: 227900:0:(client.c:2503:ptlrpc_expire_one_request()) Skipped 2 previous similar messages [ 6643.744843] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 6643.774621] Lustre: Skipped 1 previous similar message [ 6643.788443] LustreError: 228587:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 6643.819255] LustreError: 228587:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 1 previous similar message [ 6646.186752] LustreError: 228566:0:(ldlm_lockd.c:2580:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1787196919 with bad export cookie 17849665784617398719 [ 6646.189366] LustreError: MGC192.168.206.135@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 6646.211592] LustreError: 228566:0:(ldlm_lockd.c:2580:ldlm_cancel_handler()) Skipped 1 previous similar message [ 6646.613734] Lustre: server umount lustre-MDT0001 complete [ 6659.579864] Lustre: DEBUG MARKER: oleg635-server.virtnet: executing unload_modules_local [ 6662.795895] Key type lgssc unregistered [ 6663.166783] LNet: 233798:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 6663.196988] LNetError: 233798:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 6664.230636] LNet: Removed LNI 192.168.206.135@tcp [ 6665.437127] Key type .llcrypt unregistered [ 6665.438860] Key type ._llcrypt unregistered [ 6698.773685] Key type ._llcrypt registered [ 6698.778253] Key type .llcrypt registered [ 6698.914480] Lustre: DEBUG MARKER: oleg635-server.virtnet: executing load_modules_local [ 6700.073875] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 6700.220978] alg: No test for adler32 (adler32-zlib) [ 6701.254510] Lustre: Lustre: Build Version: 2.17.57_44_gafaaa93 [ 6701.489441] LNet: Added LNI 192.168.206.135@tcp [8/256/0/180] [ 6703.191168] Key type lgssc registered [ 6704.746321] Lustre: Echo OBD driver; http://www.lustre.org/ [ 6715.334191] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 6715.377138] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 6716.739107] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 6721.989835] Lustre: DEBUG MARKER: oleg635-server.virtnet: executing set_default_debug -1 all [ 6727.137407] LustreError: 235529:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 6727.161125] LustreError: 235529:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 2 previous similar messages [ 6731.084327] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 6731.324429] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 6736.694779] Lustre: DEBUG MARKER: oleg635-server.virtnet: executing set_default_debug -1 all [ 6740.203821] Lustre: Modifying parameter general.debug_raw_pointers=Y in log params [ 6747.399365] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 6747.826289] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 6754.439255] Lustre: DEBUG MARKER: oleg635-server.virtnet: executing set_default_debug -1 all [ 6762.166077] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:49698 to 0x280000401:49728) [ 6763.753562] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 6763.998439] Lustre: lustre-OST0001: new disk, initializing [ 6764.002959] Lustre: srv-lustre-OST0001: No data found on store. Initialize space. [ 6764.009977] Lustre: 237839:0:(osd_compat.c:1353:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 6764.139845] Lustre: lustre-OST0001: Imperative Recovery not enabled, recovery window 60-180 [ 6771.276985] Lustre: DEBUG MARKER: oleg635-server.virtnet: executing set_default_debug -1 all [ 6771.396686] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x00000002c0000400-0x0000000300000400]:1:ost [ 6771.406996] Lustre: cli-lustre-OST0001-super: Allocated super-sequence [0x00000002c0000400-0x0000000300000400]:1:ost] [ 6771.557580] Lustre: lustre-OST0001-osc-MDT0000: update sequence from 0x100010000 to 0x2c0000401 [ 6783.272596] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 6795.167310] Lustre: Setting parameter general.lod.*.mdt_hash=crush in log params [ 6812.649197] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 6812.657018] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 6812.665710] Lustre: Skipped 2 previous similar messages [ 6812.679108] Lustre: Skipped 3 previous similar messages [ 6815.795977] Lustre: server umount lustre-MDT0000 complete [ 6817.769194] LustreError: 236484:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 6817.796171] LustreError: 236484:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 3 previous similar messages [ 6819.687323] LustreError: 235511:0:(ldlm_lockd.c:2580:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1787197093 with bad export cookie 15820279929071428905 [ 6819.702165] LustreError: MGC192.168.206.135@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 6819.896664] Lustre: server umount lustre-MDT0001 complete [ 6833.396550] Lustre: server umount lustre-OST0000 complete [ 6846.569794] Lustre: server umount lustre-OST0001 complete [ 6866.041759] Lustre: DEBUG MARKER: oleg635-server.virtnet: executing load_modules_local [ 6875.187881] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 6875.559611] LustreError: 240946:0:(ldlm_lib.c:1190:target_handle_connect()) mylustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 6875.642511] Lustre: mylustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 6879.958584] Lustre: DEBUG MARKER: oleg635-server.virtnet: executing set_default_debug -1 all [ 6880.738815] LustreError: 240947:0:(ldlm_lib.c:1190:target_handle_connect()) mylustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 6885.858013] LustreError: 240946:0:(ldlm_lib.c:1190:target_handle_connect()) mylustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 6888.001698] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 6888.334958] Lustre: mylustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 6892.464432] Lustre: DEBUG MARKER: oleg635-server.virtnet: executing set_default_debug -1 all [ 6895.122624] Lustre: 242087:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify general/debug_raw_pointers=Y (mode = 0) failed: rc = -17 [ 6902.052724] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 6907.823921] Lustre: DEBUG MARKER: oleg635-server.virtnet: executing set_default_debug -1 all [ 6912.745385] LustreError: 242442:0:(ldlm_lib.c:1190:target_handle_connect()) mylustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 6912.774864] Lustre: mylustre-OST0000: new connection from mylustre-MDT0001-mdtlov (cleaning up unused objects from 0x280000400:100 to 0x280000400:129) [ 6916.560395] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 6916.818524] Lustre: mylustre-OST0001: Imperative Recovery not enabled, recovery window 60-180 [ 6916.828980] Lustre: Skipped 1 previous similar message [ 6920.887327] Lustre: mylustre-OST0000: new connection from mylustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:49698 to 0x280000401:49760) [ 6923.088513] Lustre: DEBUG MARKER: oleg635-server.virtnet: executing set_default_debug -1 all [ 6930.031463] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 6933.165785] Lustre: 243958:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify general/lod.*.mdt_hash=crush (mode = 0) failed: rc = -17 [ 6944.743597] LustreError: mylustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 6944.755083] Lustre: mylustre-MDT0000-osp-MDT0001: Connection to mylustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 6944.777703] Lustre: Skipped 1 previous similar message [ 6944.786818] Lustre: mylustre-MDT0000: Not available for connect from 0@lo (stopping) [ 6946.787601] Lustre: mylustre-MDT0000: Not available for connect from 0@lo (stopping) [ 6946.794099] Lustre: mylustre-MDT0000-lwp-OST0000: Connection to mylustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 6946.815139] Lustre: Skipped 2 previous similar messages [ 6948.787767] Lustre: server umount mylustre-MDT0000 complete [ 6951.907222] LustreError: 240946:0:(ldlm_lib.c:1190:target_handle_connect()) mylustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 6951.923237] LustreError: 240946:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 3 previous similar messages [ 6952.663771] LustreError: 240921:0:(ldlm_lockd.c:2580:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1787197226 with bad export cookie 15820279929071431768 [ 6952.678832] LustreError: MGC192.168.206.135@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 6952.682813] LustreError: 240921:0:(ldlm_lockd.c:2580:ldlm_cancel_handler()) Skipped 8 previous similar messages [ 6952.871985] Lustre: server umount mylustre-MDT0001 complete [ 6967.360924] Lustre: server umount mylustre-OST0000 complete [ 6981.186361] Lustre: server umount mylustre-OST0001 complete [ 6998.757233] Lustre: DEBUG MARKER: oleg635-server.virtnet: executing load_modules_local [ 7006.382296] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 7006.674887] LustreError: 246268:0:(ldlm_lib.c:1190:target_handle_connect()) tfs-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 7006.755762] Lustre: tfs-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 7010.383197] Lustre: DEBUG MARKER: oleg635-server.virtnet: executing set_default_debug -1 all [ 7018.003957] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 7022.614705] Lustre: DEBUG MARKER: oleg635-server.virtnet: executing set_default_debug -1 all [ 7025.737894] Lustre: 247412:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify general/debug_raw_pointers=Y (mode = 0) failed: rc = -17 [ 7033.554818] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 7041.760271] Lustre: DEBUG MARKER: oleg635-server.virtnet: executing set_default_debug -1 all [ 7042.306340] Lustre: tfs-OST0000: new connection from tfs-MDT0001-mdtlov (cleaning up unused objects from 0x280000400:131 to 0x280000400:161) [ 7051.119981] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 7052.407688] Lustre: tfs-OST0000: new connection from tfs-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:49698 to 0x280000401:49792) [ 7056.996267] Lustre: DEBUG MARKER: oleg635-server.virtnet: executing set_default_debug -1 all [ 7064.121141] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 7067.805542] Lustre: 249284:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify general/lod.*.mdt_hash=crush (mode = 0) failed: rc = -17 [ 7074.785839] LustreError: tfs-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 7074.794173] Lustre: tfs-MDT0000-osp-MDT0001: Connection to tfs-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 7074.801931] Lustre: tfs-MDT0000: Not available for connect from 0@lo (stopping) [ 7074.804854] Lustre: Skipped 3 previous similar messages [ 7079.636299] LustreError: 249533:0:(ldlm_resource.c:1166:ldlm_resource_complain()) MGS: namespace resource [0x736674:0x0:0x0].0x0 (ffff968c23626c00) refcount nonzero (1) after lock cleanup; forcing cleanup. [ 7079.684394] Lustre: server umount tfs-MDT0000 complete [ 7083.064732] LustreError: 246243:0:(ldlm_lockd.c:2580:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1787197356 with bad export cookie 15820279929071433721 [ 7083.079508] LustreError: MGC192.168.206.135@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 7083.287567] Lustre: server umount tfs-MDT0001 complete [ 7098.120584] Lustre: server umount tfs-OST0000 complete [ 7099.344149] Lustre: 234793:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1787197357/real 1787197357] req@ffff968d341bb100 x1874011855468544/t0(0) o400->tfs-MDT0001-lwp-OST0001@0@lo:12/10 lens 224/224 e 0 to 1 dl 1787197373 ref 1 fl Rpc:XNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 [ 7099.392892] Lustre: tfs-MDT0001-lwp-OST0001: Connection to tfs-MDT0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 7099.419085] Lustre: Skipped 3 previous similar messages [ 7102.051306] Lustre: server umount tfs-OST0001 complete [ 7119.498649] Lustre: DEBUG MARKER: oleg635-server.virtnet: executing load_modules_local [ 7128.224534] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 7128.570875] LustreError: 251533:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 7128.588077] LustreError: 251533:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 4 previous similar messages [ 7128.662186] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 7128.672436] Lustre: Skipped 3 previous similar messages [ 7132.527829] Lustre: DEBUG MARKER: oleg635-server.virtnet: executing set_default_debug -1 all [ 7139.469146] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 7142.975760] Lustre: DEBUG MARKER: oleg635-server.virtnet: executing set_default_debug -1 all [ 7145.248784] Lustre: 252675:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify general/debug_raw_pointers=Y (mode = 0) failed: rc = -17 [ 7150.391503] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 7153.777445] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:49698 to 0x280000401:49824) [ 7156.043324] Lustre: DEBUG MARKER: oleg635-server.virtnet: executing set_default_debug -1 all [ 7162.079761] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 7164.344697] Lustre: lustre-OST0000: new connection from lustre-MDT0001-mdtlov (cleaning up unused objects from 0x280000400:131 to 0x280000400:193) [ 7167.044776] Lustre: DEBUG MARKER: oleg635-server.virtnet: executing set_default_debug -1 all [ 7173.749616] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 7176.795309] Lustre: 254544:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify general/lod.*.mdt_hash=crush (mode = 0) failed: rc = -17 [ 7182.562132] Lustre: DEBUG MARKER: == conf-sanity test 104a: Make sure user defined options are reflected in mount ========================================================== 23:44:14 (1787197454) [ 7200.229387] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 7200.232232] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 7200.247885] Lustre: Skipped 3 previous similar messages [ 7200.258562] Lustre: Skipped 7 previous similar messages [ 7202.983773] Lustre: server umount lustre-MDT0000 complete [ 7205.952069] LustreError: 251510:0:(ldlm_lockd.c:2580:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1787197479 with bad export cookie 15820279929071435415 [ 7205.959755] LustreError: MGC192.168.206.135@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 7205.965762] LustreError: 251510:0:(ldlm_lockd.c:2580:ldlm_cancel_handler()) Skipped 7 previous similar messages [ 7210.466339] Lustre: lustre-MDT0001: Not available for connect from 0@lo (stopping) [ 7210.475366] Lustre: Skipped 1 previous similar message [ 7212.209878] Lustre: server umount lustre-MDT0001 complete [ 7221.398340] Lustre: server umount lustre-OST0000 complete [ 7230.502059] Lustre: server umount lustre-OST0001 complete [ 7235.146279] Lustre: DEBUG MARKER: oleg635-server.virtnet: executing set_hostid [ 7242.847569] Lustre: DEBUG MARKER: oleg635-server.virtnet: executing load_modules_local [ 7269.333098] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 7269.496753] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 7269.522348] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 7269.529749] Lustre: lustre-MDT0000: new disk, initializing [ 7269.605430] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 188-180 [ 7269.614318] Lustre: Skipped 3 previous similar messages [ 7269.633046] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 7272.763425] Lustre: DEBUG MARKER: oleg635-server.virtnet: executing set_default_debug -1 all [ 7280.095282] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 7280.172931] Lustre: 257724:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 7280.221294] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [ 7280.226211] Lustre: Skipped 1 previous similar message [ 7280.237902] Lustre: lustre-MDT0001: new disk, initializing [ 7280.346937] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [ 7280.352996] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 7283.283471] Lustre: DEBUG MARKER: oleg635-server.virtnet: executing set_default_debug -1 all [ 7291.125545] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 7291.310371] Lustre: lustre-OST0000: new disk, initializing [ 7291.313296] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 7291.318504] Lustre: 259306:0:(osd_compat.c:1353:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 7293.083826] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [ 7293.098329] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 7293.197864] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 7296.505386] Lustre: DEBUG MARKER: oleg635-server.virtnet: executing set_default_debug -1 all [ 7305.954460] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 7306.087534] Lustre: lustre-OST0001: new disk, initializing [ 7306.091294] Lustre: srv-lustre-OST0001: No data found on store. Initialize space. [ 7306.097044] Lustre: 260177:0:(osd_compat.c:1353:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 7307.272125] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x00000002c0000400-0x0000000300000400]:1:ost [ 7307.286640] Lustre: cli-lustre-OST0001-super: Allocated super-sequence [0x00000002c0000400-0x0000000300000400]:1:ost] [ 7307.373458] Lustre: lustre-OST0001-osc-MDT0000: update sequence from 0x100010000 to 0x2c0000401 [ 7310.994164] Lustre: DEBUG MARKER: oleg635-server.virtnet: executing set_default_debug -1 all [ 7321.843615] Lustre: DEBUG MARKER: == conf-sanity test 104b: Mount uses last flock argument ========================================================== 23:46:33 (1787197593) [ 7335.965297] Lustre: DEBUG MARKER: == conf-sanity test 105: check file creation for ro and rw bind mnt pt ========================================================== 23:46:47 (1787197607) [ 7353.319189] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 7353.327132] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 7353.342247] Lustre: Skipped 3 previous similar messages [ 7356.219736] Lustre: server umount lustre-OST0000 complete [ 7358.443647] LustreError: 259291:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-OST0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 7358.470762] LustreError: 259291:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 11 previous similar messages [ 7362.527917] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 7365.764154] Lustre: server umount lustre-MDT0000 complete [ 7369.074182] LustreError: 257719:0:(ldlm_lockd.c:2580:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1787197642 with bad export cookie 15820279929071436969 [ 7369.078457] LustreError: MGC192.168.206.135@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 7369.087653] LustreError: 257719:0:(ldlm_lockd.c:2580:ldlm_cancel_handler()) Skipped 2 previous similar messages [ 7375.289625] Lustre: server umount lustre-MDT0001 complete [ 7383.853954] Lustre: DEBUG MARKER: oleg635-server.virtnet: executing unload_modules_local [ 7453.351754] Lustre: server umount lustre-OST0001 complete [ 7458.626391] Lustre: DEBUG MARKER: oleg635-server.virtnet: executing set_hostid [ 7470.757535] Lustre: DEBUG MARKER: oleg635-server.virtnet: executing load_modules_local [ 7501.153911] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 7501.355203] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 7501.401710] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 7501.407956] Lustre: lustre-MDT0000: new disk, initializing [ 7501.500719] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 7504.720576] Lustre: DEBUG MARKER: oleg635-server.virtnet: executing set_default_debug -1 all [ 7513.887731] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 7514.019079] Lustre: 265004:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 7514.172367] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 7517.922409] Lustre: DEBUG MARKER: oleg635-server.virtnet: executing set_default_debug -1 all [ 7526.390517] Lustre: DEBUG MARKER: oleg635-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 7531.530684] Lustre: DEBUG MARKER: oleg635-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 7537.041613] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 7537.247880] Lustre: lustre-OST0000: new disk, initializing [ 7537.250947] Lustre: Skipped 1 previous similar message [ 7537.254082] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 7537.256657] Lustre: Skipped 2 previous similar messages [ 7537.259247] Lustre: 266972:0:(osd_compat.c:1353:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 7537.332206] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 7537.338899] Lustre: Skipped 5 previous similar messages [ 7538.482912] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [ 7538.495080] Lustre: Skipped 1 previous similar message [ 7538.503822] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 7538.584590] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 7542.522362] Lustre: DEBUG MARKER: oleg635-server.virtnet: executing set_default_debug -1 all [ 7550.811974] Lustre: DEBUG MARKER: oleg635-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 7559.142964] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 7559.144519] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 7559.149588] LustreError: Skipped 1 previous similar message [ 7559.151648] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 7559.168139] Lustre: Skipped 5 previous similar messages [ 7559.179415] Lustre: Skipped 8 previous similar messages [ 7562.294245] Lustre: server umount lustre-OST0000 complete [ 7572.679472] Lustre: server umount lustre-MDT0000 complete [ 7577.059654] LustreError: 264997:0:(ldlm_lockd.c:2580:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1787197850 with bad export cookie 15820279929071438439 [ 7577.065640] LustreError: MGC192.168.206.135@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 7577.067880] LustreError: 264997:0:(ldlm_lockd.c:2580:ldlm_cancel_handler()) Skipped 1 previous similar message [ 7577.350838] Lustre: server umount lustre-MDT0001 complete [ 7587.775539] Lustre: DEBUG MARKER: oleg635-server.virtnet: executing unload_modules_local [ 7590.970737] Key type lgssc unregistered [ 7591.263907] LNet: 268737:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 7591.273836] LNetError: 268737:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 7591.306575] LNet: Removed LNI 192.168.206.135@tcp [ 7592.148150] Key type .llcrypt unregistered [ 7592.151783] Key type ._llcrypt unregistered [ 7614.315223] Key type ._llcrypt registered [ 7614.317767] Key type .llcrypt registered [ 7614.404664] Lustre: DEBUG MARKER: oleg635-server.virtnet: executing load_modules_local [ 7615.165455] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 7615.242804] alg: No test for adler32 (adler32-zlib) [ 7616.305232] Lustre: Lustre: Build Version: 2.17.57_44_gafaaa93 [ 7616.523112] LNet: Added LNI 192.168.206.135@tcp [8/256/0/180] [ 7618.151222] Key type lgssc registered [ 7619.182284] Lustre: Echo OBD driver; http://www.lustre.org/ [ 7625.742554] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 7625.762619] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 7627.151993] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 7630.344552] Lustre: DEBUG MARKER: oleg635-server.virtnet: executing set_default_debug -1 all [ 7636.455086] LustreError: 270199:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 7636.467391] LustreError: 270199:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 2 previous similar messages [ 7636.937093] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 7637.158928] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 7640.339329] Lustre: DEBUG MARKER: oleg635-server.virtnet: executing set_default_debug -1 all [ 7646.227224] Lustre: DEBUG MARKER: oleg635-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 7651.517815] Lustre: DEBUG MARKER: oleg635-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 7657.295552] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 7657.636242] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 7662.688758] Lustre: DEBUG MARKER: oleg635-server.virtnet: executing set_default_debug -1 all [ 7668.445224] Lustre: DEBUG MARKER: oleg635-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 7669.269783] Lustre: Setting parameter lustre-OST0000.ost.unknown_param=50 in log lustre-OST0000 [ 7671.652290] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 7671.656070] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 7676.815269] Lustre: server umount lustre-OST0000 complete [ 7682.027865] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 7682.038759] Lustre: Skipped 1 previous similar message [ 7682.040204] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 7682.050920] Lustre: Skipped 2 previous similar messages [ 7687.137418] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 7687.139878] Lustre: Skipped 1 previous similar message [ 7692.257450] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 7692.267645] Lustre: Skipped 1 previous similar message [ 7694.303119] Lustre: lustre-MDT0000 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 7694.380649] Lustre: server umount lustre-MDT0000 complete [ 7697.379415] LustreError: 270200:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 7697.398295] LustreError: 270200:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 1 previous similar message [ 7697.683442] LustreError: 270180:0:(ldlm_lockd.c:2580:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1787197971 with bad export cookie 16778184415693348894 [ 7697.687735] LustreError: MGC192.168.206.135@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 7697.698311] LustreError: 270180:0:(ldlm_lockd.c:2580:ldlm_cancel_handler()) Skipped 2 previous similar messages [ 7697.910347] Lustre: server umount lustre-MDT0001 complete [ 7706.097540] Lustre: DEBUG MARKER: oleg635-server.virtnet: executing unload_modules_local [ 7708.575643] Key type lgssc unregistered [ 7708.780735] LNet: 273789:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 7708.786955] LNetError: 273789:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 7708.800528] LNet: Removed LNI 192.168.206.135@tcp [ 7709.401903] Key type .llcrypt unregistered [ 7709.406052] Key type ._llcrypt unregistered [ 7724.588224] Key type ._llcrypt registered [ 7724.590092] Key type .llcrypt registered [ 7724.706290] Lustre: DEBUG MARKER: oleg635-server.virtnet: executing load_modules_local [ 7725.709084] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 7725.777354] alg: No test for adler32 (adler32-zlib) [ 7726.895675] Lustre: Lustre: Build Version: 2.17.57_44_gafaaa93 [ 7727.159235] LNet: Added LNI 192.168.206.135@tcp [8/256/0/180] [ 7728.831249] Key type lgssc registered [ 7730.076596] Lustre: Echo OBD driver; http://www.lustre.org/ [ 7738.540670] Lustre: lustre-OST0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 7738.580898] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 7754.207825] LustreError: Failed to get MGS log lustre-sptlrpc and no local copy. [ 7759.327552] LustreError: 274983:0:(mgc_request.c:233:do_config_log_add()) MGC192.168.206.135@tcp: failed processing log, type 4: rc = -110 [ 7785.951176] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 7791.142343] Lustre: DEBUG MARKER: oleg635-server.virtnet: executing set_default_debug -1 all [ 7797.274214] Lustre: DEBUG MARKER: oleg635-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 7798.752961] Lustre: server umount lustre-OST0000 complete [ 7810.584242] Lustre: DEBUG MARKER: oleg635-server.virtnet: executing unload_modules_local [ 7812.758680] Key type lgssc unregistered [ 7812.950870] LNet: 276617:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 7812.958237] LNetError: 276617:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 7812.970769] LNet: Removed LNI 192.168.206.135@tcp [ 7813.520948] Key type .llcrypt unregistered [ 7813.526164] Key type ._llcrypt unregistered [ 7846.684847] Key type ._llcrypt registered [ 7846.686866] Key type .llcrypt registered [ 7846.749053] Lustre: DEBUG MARKER: oleg635-server.virtnet: executing load_modules_local [ 7847.589522] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 7847.722394] alg: No test for adler32 (adler32-zlib) [ 7848.787691] Lustre: Lustre: Build Version: 2.17.57_44_gafaaa93 [ 7848.993049] LNet: Added LNI 192.168.206.135@tcp [8/256/0/180] [ 7850.639181] Key type lgssc registered [ 7851.641499] Lustre: Echo OBD driver; http://www.lustre.org/ [ 7862.313894] loop0: detected capacity change from 0 to 204800000 [ 7862.464669] blk_update_request: operation not supported error, dev loop0, sector 399872 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [ 7862.476863] blk_update_request: operation not supported error, dev loop0, sector 208 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [ 7862.695275] blk_update_request: operation not supported error, dev loop0, sector 160232 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [ 7863.585369] loop0: detected capacity change from 0 to 204800000 [ 7863.762975] blk_update_request: operation not supported error, dev loop0, sector 399872 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [ 7863.776822] blk_update_request: operation not supported error, dev loop0, sector 208 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [ 7864.055430] blk_update_request: operation not supported error, dev loop0, sector 160232 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [ 7864.873492] loop0: detected capacity change from 0 to 204800000 [ 7864.971639] blk_update_request: operation not supported error, dev loop0, sector 399872 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [ 7864.981082] blk_update_request: operation not supported error, dev loop0, sector 8224 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [ 7865.071957] blk_update_request: operation not supported error, dev loop0, sector 58264 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [ 7865.928883] loop0: detected capacity change from 0 to 204800000 [ 7866.089808] blk_update_request: operation not supported error, dev loop0, sector 399872 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [ 7866.973202] loop0: detected capacity change from 0 to 204800000 [ 7866.992429] LDISKFS-fs (loop0): mounted filesystem with ordered data mode. Opts: (null) [ 7870.556819] loop0: detected capacity change from 0 to 204800000 [ 7870.585322] LDISKFS-fs (loop0): mounted filesystem with ordered data mode. Opts: (null) [ 7874.118401] loop0: detected capacity change from 0 to 204800000 [ 7874.145358] LDISKFS-fs (loop0): mounted filesystem with ordered data mode. Opts: (null) [ 7877.421526] loop0: detected capacity change from 0 to 204800000 [ 7877.442553] LDISKFS-fs (loop0): mounted filesystem with ordered data mode. Opts: (null) [ 7880.711838] loop0: detected capacity change from 0 to 204800000 [ 7880.801043] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 7880.861839] LDISKFS-fs (loop0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 7880.925309] Lustre: 280030:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0000: restore index 'fld' with [0x200000001:0x3:0x0]: rc = 0 [ 7880.932871] Lustre: 280030:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0000: restore index 'nodemap' with [0x200000003:0x3:0x0]: rc = 0 [ 7880.940338] Lustre: 280030:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0000: restore index '0x1010000-MDT0000' with [0x200000005:0x2:0x0]: rc = 0 [ 7880.947616] Lustre: 280030:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0000: restore index '0x2010000-MDT0000' with [0x200000005:0x3:0x0]: rc = 0 [ 7880.956949] Lustre: 280030:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0000: restore index '0x2010000' with [0x200000003:0xc:0x0]: rc = 0 [ 7880.974044] Lustre: 280030:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0000: restore index '0x10000' with [0x200000003:0xa:0x0]: rc = 0 [ 7880.983352] Lustre: 280030:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0000: restore index '0x10000-MDT0000' with [0x200000005:0x1:0x0]: rc = 0 [ 7880.992092] Lustre: 280030:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0000: restore index '0x1010000' with [0x200000003:0xb:0x0]: rc = 0 [ 7880.999937] Lustre: 280030:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0000: restore index '0x2010000' with [0x200000006:0x2010000:0x0]: rc = 0 [ 7881.015358] Lustre: 280030:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0000: restore index '0x10000' with [0x200000006:0x10000:0x0]: rc = 0 [ 7881.029301] Lustre: 280030:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0000: restore index '0x1010000' with [0x200000006:0x1010000:0x0]: rc = 0 [ 7881.040613] Lustre: 280030:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0000: restore index '0x2020000' with [0x200000006:0x2020000:0x0]: rc = 0 [ 7881.054550] Lustre: 280030:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0000: restore index '0x1020000' with [0x200000006:0x1020000:0x0]: rc = 0 [ 7881.067525] Lustre: 280030:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0000: restore index '0x20000' with [0x200000006:0x20000:0x0]: rc = 0 [ 7885.902300] Lustre: server umount MGS complete [ 7886.512113] loop0: detected capacity change from 0 to 204800000 [ 7886.566113] LDISKFS-fs (loop0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 7886.844542] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 7886.878542] Lustre: lustre-MDT0000: Aborting client recovery [ 7887.543567] loop1: detected capacity change from 0 to 204800000 [ 7887.663940] LDISKFS-fs (loop1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 7887.736800] Lustre: 280461:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0001: restore index 'fld' with [0x200000001:0x3:0x0]: rc = 0 [ 7887.751120] Lustre: 280461:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0001: restore index '0x1010000-MDT0001' with [0x200000005:0x2:0x0]: rc = 0 [ 7887.760765] Lustre: 280461:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0001: restore index '0x2010000' with [0x200000003:0x6:0x0]: rc = 0 [ 7887.774308] Lustre: 280461:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0001: restore index '0x10000-MDT0001' with [0x200000005:0x1:0x0]: rc = 0 [ 7887.792214] Lustre: 280461:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0001: restore index '0x2010000-MDT0001' with [0x200000005:0x3:0x0]: rc = 0 [ 7887.803812] Lustre: 280461:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0001: restore index '0x1010000' with [0x200000003:0x5:0x0]: rc = 0 [ 7887.816832] Lustre: 280461:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0001: restore index '0x10000' with [0x200000003:0x4:0x0]: rc = 0 [ 7887.836135] Lustre: Found index 1 for lustre-MDT0001, updating log [ 7887.915127] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 7887.979050] Lustre: lustre-MDT0001: Aborting client recovery [ 7888.736245] loop2: detected capacity change from 0 to 204800000 [ 7888.914873] LDISKFS-fs (loop2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 7888.955445] Lustre: 280541:0:(osd_scrub.c:1854:osd_index_restore()) lustre-OST0000: restore index 'fld' with [0x200000001:0x3:0x0]: rc = 0 [ 7888.966638] Lustre: 280541:0:(osd_scrub.c:1854:osd_index_restore()) lustre-OST0000: restore index '0x2020000' with [0x200000003:0x7:0x0]: rc = 0 [ 7888.974498] Lustre: 280541:0:(osd_scrub.c:1854:osd_index_restore()) lustre-OST0000: restore index '0x1020000-OST0000' with [0x200000005:0x2:0x0]: rc = 0 [ 7888.982778] Lustre: 280541:0:(osd_scrub.c:1854:osd_index_restore()) lustre-OST0000: restore index '0x20000-OST0000' with [0x200000005:0x1:0x0]: rc = 0 [ 7888.993259] Lustre: 280541:0:(osd_scrub.c:1854:osd_index_restore()) lustre-OST0000: restore index '0x2020000-OST0000' with [0x200000005:0x3:0x0]: rc = 0 [ 7889.002942] Lustre: 280541:0:(osd_scrub.c:1854:osd_index_restore()) lustre-OST0000: restore index '0x1020000' with [0x200000003:0x6:0x0]: rc = 0 [ 7889.013765] Lustre: 280541:0:(osd_scrub.c:1854:osd_index_restore()) lustre-OST0000: restore index '0x20000' with [0x200000003:0x5:0x0]: rc = 0 [ 7889.123509] Lustre: Found index 0 for lustre-OST0000, updating log [ 7889.141872] Lustre: 280560:0:(osd_compat.c:1353:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 7889.189754] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 7889.206392] Lustre: lustre-OST0000: Aborting recovery [ 7889.894397] loop3: detected capacity change from 0 to 204800000 [ 7889.982846] LDISKFS-fs (loop3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 7890.030629] Lustre: 280626:0:(osd_scrub.c:1854:osd_index_restore()) lustre-OST0001: restore index 'fld' with [0x200000001:0x3:0x0]: rc = 0 [ 7890.040083] Lustre: 280626:0:(osd_scrub.c:1854:osd_index_restore()) lustre-OST0001: restore index '0x20000-OST0001' with [0x200000005:0x1:0x0]: rc = 0 [ 7890.053280] Lustre: 280626:0:(osd_scrub.c:1854:osd_index_restore()) lustre-OST0001: restore index '0x1020000' with [0x200000003:0x6:0x0]: rc = 0 [ 7890.063422] Lustre: 280626:0:(osd_scrub.c:1854:osd_index_restore()) lustre-OST0001: restore index '0x1020000-OST0001' with [0x200000005:0x2:0x0]: rc = 0 [ 7890.080449] Lustre: 280626:0:(osd_scrub.c:1854:osd_index_restore()) lustre-OST0001: restore index '0x20000' with [0x200000003:0x5:0x0]: rc = 0 [ 7890.091763] Lustre: 280626:0:(osd_scrub.c:1854:osd_index_restore()) lustre-OST0001: restore index '0x2020000-OST0001' with [0x200000005:0x3:0x0]: rc = 0 [ 7890.102991] Lustre: 280626:0:(osd_scrub.c:1854:osd_index_restore()) lustre-OST0001: restore index '0x2020000' with [0x200000003:0x7:0x0]: rc = 0 [ 7890.129390] Lustre: 280632:0:(osd_compat.c:1353:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 7890.187775] Lustre: lustre-OST0001: Aborting recovery [ 7895.543239] LustreError: 280405:0:(lod_dev.c:511:lod_sub_recovery_thread()) lustre-MDT0001-osp-MDT0000: get update log duration 9, retries 0, failed: rc = -108 [ 7896.551127] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x0:6 to 0x0:33) [ 7896.554766] Lustre: lustre-OST0001: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x0:6 to 0x0:33) [ 7896.608914] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000bd0 [ 7896.622389] Lustre: lustre-OST0001-osc-MDT0000: update sequence from 0x100010000 to 0x2c0000bd0 [ 7897.573616] Lustre: lustre-OST0001: new connection from lustre-MDT0001-mdtlov (cleaning up unused objects from 0x2c0000400:5 to 0x2c0000400:33) [ 7897.576143] Lustre: lustre-OST0000: new connection from lustre-MDT0001-mdtlov (cleaning up unused objects from 0x280000400:5 to 0x280000400:33) [ 7898.357517] LustreError: 280372:0:(mdt_identity.c:115:mdt_identity_do_upcall()) lustre-MDT0000: error invoking upcall /root/Work/Lustre/R5/lustre-release/lustre/tests/../utils/l_getidentity lustre-MDT0000 0: rc -2; check /proc/fs/lustre/mdt/lustre-MDT0000/identity_upcall, time 591us: rc = -2 [ 7898.875328] LustreError: 280373:0:(mdt_identity.c:115:mdt_identity_do_upcall()) lustre-MDT0001: error invoking upcall /root/Work/Lustre/R5/lustre-release/lustre/tests/../utils/l_getidentity lustre-MDT0001 0: rc -2; check /proc/fs/lustre/mdt/lustre-MDT0001/identity_upcall, time 4493us: rc = -2 [ 7898.898027] LustreError: 280373:0:(mdt_identity.c:115:mdt_identity_do_upcall()) Skipped 27 previous similar messages [ 7912.933404] Lustre: lustre-MDT0000-lwp-OST0000: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 7912.942058] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 7912.960781] Lustre: Skipped 3 previous similar messages [ 7912.981658] Lustre: Skipped 3 previous similar messages [ 7917.345766] Lustre: server umount lustre-MDT0000 complete [ 7918.053330] LustreError: 280370:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 7918.068836] LustreError: 280370:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 5 previous similar messages [ 7918.834576] LustreError: 280354:0:(ldlm_lockd.c:2580:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1787198192 with bad export cookie 3881415976163150310 [ 7918.835385] LustreError: MGC192.168.206.135@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 7918.848135] LustreError: 280354:0:(ldlm_lockd.c:2580:ldlm_cancel_handler()) Skipped 1 previous similar message [ 7923.168401] Lustre: lustre-MDT0001-lwp-OST0000: Connection to lustre-MDT0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 7923.169310] LustreError: 280371:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 7923.176301] Lustre: lustre-MDT0001: Not available for connect from 0@lo (stopping) [ 7923.179273] Lustre: Skipped 1 previous similar message [ 7923.209882] LustreError: 280371:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 1 previous similar message [ 7925.050400] Lustre: server umount lustre-MDT0001 complete [ 7932.140208] Lustre: server umount lustre-OST0000 complete [ 7932.892460] LustreError: 281118:0:(ldlm_resource.c:1166:ldlm_resource_complain()) filter-lustre-OST0001_UUID: namespace resource [0x4:0x0:0x0].0x0 (00000000f13670f0) refcount nonzero (1) after lock cleanup; forcing cleanup. [ 7947.232105] Lustre: lustre-OST0001 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 7947.442328] Lustre: server umount lustre-OST0001 complete [ 7955.610645] Lustre: DEBUG MARKER: == conf-sanity test 109a: test lctl clear_conf fsname ==== 23:57:07 (1787198227) [ 7980.500572] Lustre: DEBUG MARKER: oleg635-server.virtnet: executing set_hostid [ 7989.176690] Lustre: DEBUG MARKER: oleg635-server.virtnet: executing load_modules_local [ 8016.847889] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 8017.040182] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 8017.058626] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 8017.062845] Lustre: lustre-MDT0000: new disk, initializing [ 8017.107029] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 8017.111214] Lustre: Skipped 1 previous similar message [ 8017.122575] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 8020.492967] Lustre: DEBUG MARKER: oleg635-server.virtnet: executing set_default_debug -1 all [ 8028.641432] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 8028.733841] Lustre: 284694:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 8028.787990] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [ 8028.794406] Lustre: Skipped 1 previous similar message [ 8028.800528] Lustre: lustre-MDT0001: new disk, initializing [ 8028.877830] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 8028.904627] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [ 8028.918776] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 8032.727336] Lustre: DEBUG MARKER: oleg635-server.virtnet: executing set_default_debug -1 all [ 8040.454910] Lustre: DEBUG MARKER: oleg635-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 8044.991177] Lustre: DEBUG MARKER: oleg635-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 8050.304580] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 8050.497659] Lustre: lustre-OST0000: new disk, initializing [ 8050.504690] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 8050.510252] Lustre: 286659:0:(osd_compat.c:1353:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 8050.568757] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 8055.541756] Lustre: DEBUG MARKER: oleg635-server.virtnet: executing set_default_debug -1 all [ 8059.968623] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [ 8059.980900] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 8060.083072] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 8063.777617] Lustre: DEBUG MARKER: oleg635-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 8069.527162] Lustre: Setting parameter lustre-MDT0000.mdd.atime_diff=62 in log lustre-MDT0000 [ 8082.701350] Lustre: Modifying parameter lustre-MDT0000.mdd.atime_diff=63 in log lustre-MDT0000 [ 8086.238947] Lustre: Setting parameter lustre-client.llite.max_read_ahead_mb=32 in log lustre-client [ 8092.848835] Lustre: Modifying parameter lustre-client.llite.max_read_ahead_mb=64 in log lustre-client [ 8137.187549] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 8137.190911] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 8137.198290] LustreError: Skipped 1 previous similar message [ 8137.199627] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 8137.210166] Lustre: Skipped 1 previous similar message [ 8137.230023] Lustre: Skipped 2 previous similar messages [ 8141.466327] Lustre: server umount lustre-OST0000 complete [ 8146.918631] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 8146.931055] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 8146.948507] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 8150.755606] Lustre: server umount lustre-MDT0000 complete [ 8152.548730] LustreError: 284706:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 8152.560326] LustreError: 284706:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 1 previous similar message [ 8153.598465] LustreError: 284686:0:(ldlm_lockd.c:2580:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1787198427 with bad export cookie 3881415976163153026 [ 8153.601946] LustreError: MGC192.168.206.135@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 8153.606642] LustreError: 284686:0:(ldlm_lockd.c:2580:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 8153.751854] Lustre: server umount lustre-MDT0001 complete [ 8160.936375] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 8164.739344] Lustre: DEBUG MARKER: oleg635-server.virtnet: executing set_default_debug -1 all [ 8169.456344] Lustre: server umount MGS complete [ 8179.085835] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 8179.361248] LustreError: 290529:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 8179.379949] LustreError: 290542:0:(qmt_pool.c:1639:qmt_pool_add_rem()) lustre-QMT0000: can't remove lustre-OST0000_UUID pool 'pool1': rc = -22 [ 8179.420956] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 8182.450712] Lustre: DEBUG MARKER: oleg635-server.virtnet: executing set_default_debug -1 all [ 8184.801722] LustreError: 290530:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 8190.636599] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 8195.542538] Lustre: DEBUG MARKER: oleg635-server.virtnet: executing set_default_debug -1 all [ 8201.458311] Lustre: DEBUG MARKER: oleg635-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 8207.092638] Lustre: DEBUG MARKER: oleg635-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 8212.312070] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 8212.630085] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 8212.632794] Lustre: Skipped 1 previous similar message [ 8216.824622] Lustre: DEBUG MARKER: oleg635-server.virtnet: executing set_default_debug -1 all [ 8222.131628] Lustre: DEBUG MARKER: oleg635-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 8256.993422] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 8256.994050] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 8257.001900] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 8257.001908] Lustre: Skipped 2 previous similar messages [ 8257.010187] LustreError: Skipped 1 previous similar message [ 8257.021175] Lustre: Skipped 2 previous similar messages [ 8261.701993] Lustre: server umount lustre-OST0000 complete [ 8267.234359] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 8267.246126] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 8267.250819] Lustre: Skipped 2 previous similar messages [ 8271.309703] Lustre: server umount lustre-MDT0000 complete [ 8272.354577] LustreError: 290529:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 8272.373370] LustreError: 290529:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 2 previous similar messages [ 8275.197225] LustreError: 290509:0:(ldlm_lockd.c:2580:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1787198548 with bad export cookie 3881415976163154867 [ 8275.200429] LustreError: MGC192.168.206.135@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 8275.225930] LustreError: 290509:0:(ldlm_lockd.c:2580:ldlm_cancel_handler()) Skipped 2 previous similar messages [ 8275.474620] Lustre: server umount lustre-MDT0001 complete [ 8288.049754] Lustre: DEBUG MARKER: oleg635-server.virtnet: executing unload_modules_local [ 8291.137634] Key type lgssc unregistered [ 8291.459642] LNet: 294491:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 8291.467634] LNetError: 294491:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 8292.529513] LNet: Removed LNI 192.168.206.135@tcp [ 8293.219128] Key type .llcrypt unregistered [ 8293.221926] Key type ._llcrypt unregistered [ 8328.175622] Key type ._llcrypt registered [ 8328.178570] Key type .llcrypt registered [ 8328.269749] Lustre: DEBUG MARKER: oleg635-server.virtnet: executing set_hostid [ 8341.889846] Lustre: DEBUG MARKER: oleg635-server.virtnet: executing load_modules_local [ 8342.699230] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 8342.710347] alg: No test for adler32 (adler32-zlib) [ 8343.687638] Lustre: Lustre: Build Version: 2.17.57_44_gafaaa93 [ 8343.873457] LNet: Added LNI 192.168.206.135@tcp [8/256/0/180] [ 8345.513359] Key type lgssc registered [ 8346.325227] Lustre: Echo OBD driver; http://www.lustre.org/ [ 8372.591402] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 8372.603837] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 8373.754406] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 8373.773796] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 8373.778903] Lustre: lustre-MDT0000: new disk, initializing [ 8373.830139] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 8373.842432] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 8377.031821] Lustre: DEBUG MARKER: oleg635-server.virtnet: executing set_default_debug -1 all [ 8385.464420] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 8385.550734] Lustre: 298446:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 8385.602718] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [ 8385.606330] Lustre: Skipped 1 previous similar message [ 8385.612242] Lustre: lustre-MDT0001: new disk, initializing [ 8385.693530] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 8385.727190] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [ 8385.737359] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 8389.232510] Lustre: DEBUG MARKER: oleg635-server.virtnet: executing set_default_debug -1 all [ 8396.172046] Lustre: DEBUG MARKER: oleg635-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 8400.178416] Lustre: DEBUG MARKER: oleg635-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 8405.371187] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 8405.585392] Lustre: lustre-OST0000: new disk, initializing [ 8405.589384] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 8405.598357] Lustre: 300416:0:(osd_compat.c:1353:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 8405.666426] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 8410.750726] Lustre: DEBUG MARKER: oleg635-server.virtnet: executing set_default_debug -1 all [ 8414.824637] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [ 8414.835652] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 8414.939257] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 8419.837773] Lustre: DEBUG MARKER: oleg635-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 8426.852266] Lustre: Setting parameter lustre-MDT0000.mdd.atime_diff=62 in log lustre-MDT0000 [ 8436.147450] Lustre: Modifying parameter lustre-MDT0000.mdd.atime_diff=63 in log lustre-MDT0000 [ 8443.828993] Lustre: Setting parameter lustre-client.llite.max_read_ahead_mb=32 in log lustre-client [ 8495.018641] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 8495.024817] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 8495.039027] Lustre: Skipped 1 previous similar message [ 8495.047019] Lustre: Skipped 1 previous similar message [ 8498.401244] Lustre: server umount lustre-OST0000 complete [ 8503.778958] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 8503.787186] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 8503.804561] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 8505.316161] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 8505.318171] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 8505.352542] Lustre: Skipped 1 previous similar message [ 8507.778838] Lustre: server umount lustre-MDT0000 complete [ 8511.290166] LustreError: 298439:0:(ldlm_lockd.c:2580:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1787198785 with bad export cookie 488728088068969207 [ 8511.296637] LustreError: MGC192.168.206.135@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 8511.303607] LustreError: 298439:0:(ldlm_lockd.c:2580:ldlm_cancel_handler()) Skipped 3 previous similar messages [ 8511.509464] Lustre: server umount lustre-MDT0001 complete [ 8519.260334] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 8523.277515] Lustre: DEBUG MARKER: oleg635-server.virtnet: executing set_default_debug -1 all [ 8528.255358] Lustre: server umount MGS complete [ 8541.097169] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 8541.506813] LustreError: 304281:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 8541.521420] LustreError: 304281:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 3 previous similar messages [ 8541.532174] LustreError: 304294:0:(qmt_pool.c:1639:qmt_pool_add_rem()) lustre-QMT0000: can't remove lustre-OST0000_UUID pool 'pool1': rc = -22 [ 8541.585968] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 8546.160244] Lustre: DEBUG MARKER: oleg635-server.virtnet: executing set_default_debug -1 all [ 8546.784603] LustreError: 304282:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 8551.905048] LustreError: 304281:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 8554.254070] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 8554.504199] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 8558.142767] Lustre: DEBUG MARKER: oleg635-server.virtnet: executing set_default_debug -1 all [ 8565.126138] Lustre: DEBUG MARKER: oleg635-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 8570.701209] Lustre: DEBUG MARKER: oleg635-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 8577.304731] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 8577.594529] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 8583.330184] Lustre: DEBUG MARKER: oleg635-server.virtnet: executing set_default_debug -1 all [ 8590.189757] Lustre: DEBUG MARKER: oleg635-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 8617.444334] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 8617.455001] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 8617.456614] Lustre: Skipped 1 previous similar message [ 8617.457638] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 8617.471412] Lustre: Skipped 1 previous similar message [ 8622.562841] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 8622.565426] Lustre: Skipped 1 previous similar message [ 8623.500095] Lustre: server umount lustre-OST0000 complete [ 8627.681473] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 8632.800850] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 8632.807166] Lustre: Skipped 3 previous similar messages [ 8633.352397] Lustre: server umount lustre-MDT0000 complete [ 8636.693246] LustreError: 307006:0:(ldlm_lockd.c:2580:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1787198910 with bad export cookie 488728088068971062 [ 8636.699806] LustreError: MGC192.168.206.135@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 8636.708912] LustreError: 307006:0:(ldlm_lockd.c:2580:ldlm_cancel_handler()) Skipped 2 previous similar messages [ 8636.914875] Lustre: server umount lustre-MDT0001 complete [ 8646.950611] Lustre: DEBUG MARKER: oleg635-server.virtnet: executing unload_modules_local [ 8649.389718] Key type lgssc unregistered [ 8649.724306] LNet: 308228:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 8649.745419] LNetError: 308228:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 8649.778100] LNet: Removed LNI 192.168.206.135@tcp [ 8650.663699] Key type .llcrypt unregistered [ 8650.666046] Key type ._llcrypt unregistered [ 8671.020576] Key type ._llcrypt registered [ 8671.023413] Key type .llcrypt registered [ 8671.112689] Lustre: DEBUG MARKER: oleg635-server.virtnet: executing set_hostid [ 8687.272086] Lustre: DEBUG MARKER: oleg635-server.virtnet: executing load_modules_local [ 8688.482418] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 8688.511718] alg: No test for adler32 (adler32-zlib) [ 8689.656534] Lustre: Lustre: Build Version: 2.17.57_44_gafaaa93 [ 8689.883807] LNet: Added LNI 192.168.206.135@tcp [8/256/0/180] [ 8691.568518] Key type lgssc registered [ 8692.561668] Lustre: Echo OBD driver; http://www.lustre.org/ [ 8713.819900] Lustre: DEBUG MARKER: oleg635-server.virtnet: executing load_modules_local [ 8727.317912] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 8727.394213] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 8728.615708] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 8728.647436] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 8728.654372] Lustre: lustre-MDT0000: new disk, initializing [ 8728.749474] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 8728.774656] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 8734.787323] Lustre: DEBUG MARKER: oleg635-server.virtnet: executing set_default_debug -1 all [ 8744.908861] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 8745.165724] Lustre: lustre-OST0000: new disk, initializing [ 8745.168444] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 8745.172474] Lustre: Skipped 1 previous similar message [ 8745.178432] Lustre: 312278:0:(osd_compat.c:1353:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 8745.244147] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 8750.949345] Lustre: DEBUG MARKER: oleg635-server.virtnet: executing set_default_debug -1 all [ 8754.249230] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:0:ost [ 8754.262606] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:0:ost] [ 8754.298356] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x240000400 [ 9619.424367] Lustre: lustre-MDT0000-lwp-OST0000: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 9619.447990] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 9621.476412] Lustre: server umount lustre-MDT0000 complete [ 9625.390077] LustreError: 311464:0:(ldlm_lockd.c:2580:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1787199899 with bad export cookie 8433312662624745919 [ 9625.390975] LustreError: MGC192.168.206.135@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 9625.401305] LustreError: 311464:0:(ldlm_lockd.c:2580:ldlm_cancel_handler()) Skipped 1 previous similar message [ 9625.563201] Lustre: server umount lustre-OST0000 complete [ 9650.658143] Lustre: DEBUG MARKER: oleg635-server.virtnet: executing set_hostid [ 9661.742620] Lustre: DEBUG MARKER: oleg635-server.virtnet: executing load_modules_local [ 9697.974806] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 9698.242052] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 9698.271220] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 9698.278417] Lustre: lustre-MDT0000: new disk, initializing [ 9698.340165] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 9698.353862] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 9702.576974] Lustre: DEBUG MARKER: oleg635-server.virtnet: executing set_default_debug -1 all [ 9712.411399] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 9712.544698] Lustre: 316621:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 9712.586160] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [ 9712.599364] Lustre: Skipped 1 previous similar message [ 9712.602902] Lustre: lustre-MDT0001: new disk, initializing [ 9712.686229] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 9712.731602] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [ 9712.744437] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 9716.498569] Lustre: DEBUG MARKER: oleg635-server.virtnet: executing set_default_debug -1 all [ 9723.833216] Lustre: DEBUG MARKER: oleg635-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 9729.021777] Lustre: DEBUG MARKER: oleg635-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 9735.136222] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 9735.446324] Lustre: lustre-OST0000: new disk, initializing [ 9735.450519] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 9735.454946] Lustre: 318589:0:(osd_compat.c:1353:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 9735.531946] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 9742.046591] Lustre: DEBUG MARKER: oleg635-server.virtnet: executing set_default_debug -1 all [ 9742.402285] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [ 9742.416362] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 9742.526446] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 9751.611746] Lustre: DEBUG MARKER: oleg635-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 9755.952687] Lustre: DEBUG MARKER: oleg635-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 [ 9756.177619] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [ 9760.977821] Lustre: DEBUG MARKER: oleg635-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid 50 [ 9761.208900] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [ 9763.296298] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 9763.299909] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 9763.326746] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 9768.430650] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 9768.438781] Lustre: Skipped 2 previous similar messages [ 9769.329393] Lustre: server umount lustre-OST0000 complete [ 9773.538249] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 9773.540616] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 9773.558026] Lustre: Skipped 2 previous similar messages [ 9778.659461] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 9778.665526] Lustre: Skipped 2 previous similar messages [ 9779.383440] Lustre: server umount lustre-MDT0000 complete [ 9782.934982] LustreError: 318588:0:(ldlm_lockd.c:2580:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1787200056 with bad export cookie 8433312662627686738 [ 9782.935787] LustreError: MGC192.168.206.135@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 9782.943215] LustreError: 318588:0:(ldlm_lockd.c:2580:ldlm_cancel_handler()) Skipped 1 previous similar message [ 9783.091687] Lustre: server umount lustre-MDT0001 complete [ 9786.634775] Lustre: DEBUG MARKER: SKIP: conf-sanity test_111 skipping SLOW test 111 [ 9788.525290] Lustre: DEBUG MARKER: == conf-sanity test 112a: mount OST with no_create option ========================================================== 00:27:40 (1787200060) [ 9796.094374] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 9796.524604] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 9800.829824] Lustre: DEBUG MARKER: oleg635-server.virtnet: executing set_default_debug -1 all [ 9801.699573] LustreError: 320902:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 9801.714926] LustreError: 320902:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 2 previous similar messages [ 9806.822107] LustreError: 320901:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 9809.533125] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 9809.842671] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 9813.594711] Lustre: DEBUG MARKER: oleg635-server.virtnet: executing set_default_debug -1 all [ 9821.348724] Lustre: DEBUG MARKER: oleg635-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 9827.474800] Lustre: DEBUG MARKER: oleg635-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 9834.167201] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 9834.404274] Lustre: lustre-OST0000: Not available for connect from 0@lo (not set up) [ 9834.502609] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 9840.693366] Lustre: DEBUG MARKER: oleg635-server.virtnet: executing set_default_debug -1 all [ 9848.021320] Lustre: DEBUG MARKER: oleg635-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 9854.721334] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 9854.927450] Lustre: lustre-OST0001: new disk, initializing [ 9854.933515] Lustre: srv-lustre-OST0001: No data found on store. Initialize space. [ 9854.941437] Lustre: 323738:0:(osd_compat.c:1353:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 9855.037407] Lustre: lustre-OST0001: Imperative Recovery not enabled, recovery window 60-180 [ 9861.222975] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x00000002c0000400-0x0000000300000400]:1:ost [ 9861.237098] Lustre: cli-lustre-OST0001-super: Allocated super-sequence [0x00000002c0000400-0x0000000300000400]:1:ost] [ 9861.294374] LustreError: 324075:0:(osp_precreate.c:974:osp_precreate_cleanup_orphans()) lustre-OST0001-osc-MDT0001: cannot cleanup orphans: rc = -1 [ 9861.406671] Lustre: DEBUG MARKER: oleg635-server.virtnet: executing set_default_debug -1 all [ 9862.376646] LustreError: 324075:0:(osp_precreate.c:974:osp_precreate_cleanup_orphans()) lustre-OST0001-osc-MDT0001: cannot cleanup orphans: rc = -1 [ 9862.389364] LustreError: 324075:0:(osp_precreate.c:974:osp_precreate_cleanup_orphans()) Skipped 2 previous similar messages [ 9863.403172] LustreError: 324085:0:(osp_precreate.c:974:osp_precreate_cleanup_orphans()) lustre-OST0001-osc-MDT0000: cannot cleanup orphans: rc = -1 [ 9865.507968] LustreError: 324085:0:(osp_precreate.c:974:osp_precreate_cleanup_orphans()) lustre-OST0001-osc-MDT0000: cannot cleanup orphans: rc = -1 [ 9865.519784] LustreError: 324085:0:(osp_precreate.c:974:osp_precreate_cleanup_orphans()) Skipped 4 previous similar messages [ 9869.618291] LustreError: 324075:0:(osp_precreate.c:974:osp_precreate_cleanup_orphans()) lustre-OST0001-osc-MDT0001: cannot cleanup orphans: rc = -1 [ 9869.632078] LustreError: 324075:0:(osp_precreate.c:974:osp_precreate_cleanup_orphans()) Skipped 6 previous similar messages [ 9872.096092] Lustre: DEBUG MARKER: oleg635-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 [ 9872.341278] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [ 9877.862000] LustreError: 324075:0:(osp_precreate.c:974:osp_precreate_cleanup_orphans()) lustre-OST0001-osc-MDT0001: cannot cleanup orphans: rc = -1 [ 9877.884567] LustreError: 324075:0:(osp_precreate.c:974:osp_precreate_cleanup_orphans()) Skipped 15 previous similar messages [ 9878.652526] Lustre: DEBUG MARKER: oleg635-client.virtnet: executing wait_import_state (FULL|IDLE) osc.lustre-OST0000-osc-ffff9822c567a800.ost_server_uuid 50 [ 9880.415196] Lustre: DEBUG MARKER: osc.lustre-OST0000-osc-ffff9822c567a800.ost_server_uuid in FULL state after 0 sec [ 9886.698665] Lustre: DEBUG MARKER: oleg635-client.virtnet: executing wait_import_state (FULL|IDLE) osc.lustre-OST0001-osc-ffff9822c567a800.ost_server_uuid 50 [ 9888.276743] Lustre: DEBUG MARKER: osc.lustre-OST0001-osc-ffff9822c567a800.ost_server_uuid in FULL state after 0 sec [ 9890.364409] Lustre: lustre-OST0001-osc-MDT0000: update sequence from 0x100010000 to 0x2c0000401 [ 9905.632284] LustreError: lustre-OST0001-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 9905.649546] LustreError: Skipped 1 previous similar message [ 9905.661064] Lustre: lustre-OST0001-osc-MDT0000: Connection to lustre-OST0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 9905.705365] Lustre: lustre-OST0001: Not available for connect from 0@lo (stopping) [ 9911.046258] Lustre: server umount lustre-OST0001 complete [ 9912.836033] LustreError: 322724:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 9912.868899] LustreError: 322724:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 1 previous similar message [ 9917.941193] LustreError: 322726:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 9917.968062] LustreError: 322726:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 1 previous similar message [ 9921.504259] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 9921.512387] LustreError: Skipped 1 previous similar message [ 9921.516528] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 9921.533190] Lustre: Skipped 1 previous similar message [ 9923.046734] LustreError: 322725:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 9923.065073] LustreError: 322725:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 1 previous similar message [ 9925.560909] Lustre: server umount lustre-OST0000 complete [ 9932.774985] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 9932.782639] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 9932.791685] Lustre: Skipped 1 previous similar message [ 9938.404601] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 9938.409342] Lustre: Skipped 11 previous similar messages [ 9944.031188] Lustre: lustre-MDT0000 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 9944.145416] Lustre: server umount lustre-MDT0000 complete [ 9947.620182] LustreError: 320898:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 9947.645693] LustreError: 320898:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 1 previous similar message [ 9948.381222] LustreError: 320883:0:(ldlm_lockd.c:2580:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1787200222 with bad export cookie 8433312662627687620 [ 9948.389144] LustreError: MGC192.168.206.135@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 9948.611697] Lustre: server umount lustre-MDT0001 complete [ 9958.900544] Lustre: DEBUG MARKER: oleg635-server.virtnet: executing unload_modules_local [ 9961.598840] Key type lgssc unregistered [ 9961.866595] LNet: 326430:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 9961.880756] LNetError: 326430:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 9961.898195] LNet: Removed LNI 192.168.206.135@tcp [ 9962.532826] Key type .llcrypt unregistered [ 9962.538373] Key type ._llcrypt unregistered [ 9985.236372] Key type ._llcrypt registered [ 9985.238777] Key type .llcrypt registered [ 9985.358155] Lustre: DEBUG MARKER: oleg635-server.virtnet: executing load_modules_local [ 9986.131720] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 9986.160143] alg: No test for adler32 (adler32-zlib) [ 9987.200573] Lustre: Lustre: Build Version: 2.17.57_44_gafaaa93 [ 9987.486977] LNet: Added LNI 192.168.206.135@tcp [8/256/0/180] [ 9989.184555] Key type lgssc registered [ 9990.215513] Lustre: Echo OBD driver; http://www.lustre.org/ [ 9997.527551] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 9997.554920] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 9998.934156] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [10002.964827] Lustre: DEBUG MARKER: oleg635-server.virtnet: executing set_default_debug -1 all [10009.064092] LustreError: 327844:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [10009.097536] LustreError: 327844:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 2 previous similar messages [10009.997951] Lustre: DEBUG MARKER: oleg635-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [10014.182161] LustreError: 327845:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [10016.672707] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [10016.893408] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [10021.652810] Lustre: DEBUG MARKER: oleg635-server.virtnet: executing set_default_debug -1 all [10032.254220] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [10032.530320] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [10038.283800] Lustre: DEBUG MARKER: oleg635-server.virtnet: executing set_default_debug -1 all [10041.768547] LustreError: 329525:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [10043.814568] LustreError: 330055:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [10043.832172] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:36 to 0x280000401:65) [10043.832565] LustreError: 330055:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 1 previous similar message [10045.457622] Lustre: DEBUG MARKER: oleg635-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [10048.996139] LustreError: 330055:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [10049.020412] LustreError: 330055:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 1 previous similar message [10052.671438] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [10052.995719] Lustre: lustre-OST0001: Imperative Recovery not enabled, recovery window 60-180 [10058.251206] Lustre: lustre-OST0001: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x2c0000401:3 to 0x2c0000401:33) [10058.830210] Lustre: DEBUG MARKER: oleg635-server.virtnet: executing set_default_debug -1 all [10066.443174] Lustre: DEBUG MARKER: oleg635-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0001-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [10073.059950] Lustre: DEBUG MARKER: oleg635-server.virtnet: executing wait_import_state (FULL|IDLE) os[cp].lustre-OST0001-osc-MDT0001.ost_server_uuid 50 [10073.258489] Lustre: DEBUG MARKER: os[cp].lustre-OST0001-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [10181.093966] LustreError: lustre-OST0001-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [10181.094232] Lustre: lustre-OST0001-osc-MDT0001: Connection to lustre-OST0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [10181.113325] Lustre: lustre-OST0001: Not available for connect from 0@lo (stopping) [10181.119548] LustreError: Skipped 1 previous similar message [10181.149690] Lustre: Skipped 1 previous similar message [10186.214540] Lustre: lustre-OST0001: Not available for connect from 0@lo (stopping) [10186.221379] Lustre: Skipped 2 previous similar messages [10191.333675] Lustre: lustre-OST0001: Not available for connect from 0@lo (stopping) [10191.343552] Lustre: Skipped 1 previous similar message [10194.399140] Lustre: lustre-OST0001 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [10194.506413] Lustre: server umount lustre-OST0001 complete [10196.455205] LustreError: 329527:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [10196.493203] LustreError: 329527:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 1 previous similar message [10201.571629] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [10201.577281] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [10201.597776] Lustre: Skipped 1 previous similar message [10201.618926] Lustre: Skipped 1 previous similar message [10206.691718] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [10206.701282] Lustre: Skipped 1 previous similar message [10207.645124] Lustre: server umount lustre-OST0000 complete [10211.808569] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [10216.416445] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [10216.431508] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [10216.450935] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [10216.457595] Lustre: Skipped 1 previous similar message [10217.884450] Lustre: server umount lustre-MDT0000 complete [10221.850136] LustreError: 328777:0:(ldlm_lockd.c:2580:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1787200495 with bad export cookie 6798888280149092725 [10221.852968] LustreError: MGC192.168.206.135@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [10221.858216] LustreError: 328777:0:(ldlm_lockd.c:2580:ldlm_cancel_handler()) Skipped 3 previous similar messages [10222.114468] Lustre: server umount lustre-MDT0001 complete [10231.975582] Lustre: DEBUG MARKER: oleg635-server.virtnet: executing unload_modules_local [10234.550930] Key type lgssc unregistered [10234.843075] LNet: 332791:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [10234.851136] LNetError: 332791:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [10234.886083] LNet: Removed LNI 192.168.206.135@tcp [10235.768517] Key type .llcrypt unregistered [10235.774273] Key type ._llcrypt unregistered [10277.724258] Key type ._llcrypt registered [10277.725569] Key type .llcrypt registered [10277.815570] Lustre: DEBUG MARKER: oleg635-server.virtnet: executing load_modules_local [10279.012604] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [10279.162899] alg: No test for adler32 (adler32-zlib) [10280.166673] Lustre: Lustre: Build Version: 2.17.57_44_gafaaa93 [10280.342506] LNet: Added LNI 192.168.206.135@tcp [8/256/0/180] [10282.016358] Key type lgssc registered [10283.089704] Lustre: Echo OBD driver; http://www.lustre.org/ [10291.423736] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [10291.450925] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [10292.752474] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [10296.856304] Lustre: DEBUG MARKER: oleg635-server.virtnet: executing set_default_debug -1 all [10302.982919] LustreError: 335096:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [10303.000304] LustreError: 335096:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 2 previous similar messages [10304.919107] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [10305.291957] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [10309.364964] Lustre: DEBUG MARKER: oleg635-server.virtnet: executing set_default_debug -1 all [10312.558371] Lustre: Modifying parameter general.debug_raw_pointers=Y in log params [10319.286387] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [10319.724561] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [10326.086946] Lustre: DEBUG MARKER: oleg635-server.virtnet: executing set_default_debug -1 all [10329.977600] LustreError: 337135:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [10334.283935] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [10334.504995] Lustre: lustre-OST0001: Imperative Recovery not enabled, recovery window 60-180 [10337.594314] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:36 to 0x280000401:97) [10337.601074] Lustre: lustre-OST0001: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x2c0000401:3 to 0x2c0000401:65) [10340.834241] Lustre: DEBUG MARKER: oleg635-server.virtnet: executing set_default_debug -1 all [10349.170899] Lustre: DEBUG MARKER: Using TIMEOUT=20 [10360.524391] Lustre: Setting parameter general.lod.*.mdt_hash=crush in log params [10377.190227] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [10377.208642] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [10377.224420] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [10378.720832] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [10378.722129] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [10378.731673] Lustre: Skipped 2 previous similar messages [10379.891218] Lustre: server umount lustre-MDT0000 complete [10383.665343] LustreError: 335079:0:(ldlm_lockd.c:2580:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1787200657 with bad export cookie 14374276770429827529 [10383.673450] LustreError: MGC192.168.206.135@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [10383.685230] LustreError: 335079:0:(ldlm_lockd.c:2580:ldlm_cancel_handler()) Skipped 3 previous similar messages [10383.903854] Lustre: server umount lustre-MDT0001 complete [10397.742368] Lustre: server umount lustre-OST0000 complete [10400.223220] Lustre: 334358:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1787200657/real 1787200657] req@ffff968d3b658e00 x1874015608176256/t0(0) o400->lustre-MDT0001-lwp-OST0001@0@lo:12/10 lens 224/224 e 0 to 1 dl 1787200673 ref 1 fl Rpc:XNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 [10400.242553] Lustre: lustre-MDT0001-lwp-OST0001: Connection to lustre-MDT0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [10401.634665] Lustre: server umount lustre-OST0001 complete [10424.723412] Lustre: DEBUG MARKER: oleg635-server.virtnet: executing load_modules_local [10435.819366] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [10436.132789] LustreError: 340364:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [10436.213929] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [10440.915656] Lustre: DEBUG MARKER: oleg635-server.virtnet: executing set_default_debug -1 all [10441.705858] LustreError: 340365:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [10445.800528] LustreError: 340364:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [10449.588124] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [10449.957497] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [10453.609636] Lustre: DEBUG MARKER: oleg635-server.virtnet: executing set_default_debug -1 all [10457.030259] Lustre: 341504:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify general/debug_raw_pointers=Y (mode = 0) failed: rc = -17 [10463.537713] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [10469.741555] Lustre: DEBUG MARKER: oleg635-server.virtnet: executing set_default_debug -1 all [10475.112094] LustreError: 341860:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [10478.667987] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [10478.857584] Lustre: lustre-OST0001: Imperative Recovery not enabled, recovery window 60-180 [10478.861888] Lustre: Skipped 1 previous similar message [10480.957417] Lustre: lustre-OST0001: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x2c0000401:3 to 0x2c0000401:97) [10480.989626] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:36 to 0x280000401:129) [10484.515427] Lustre: DEBUG MARKER: oleg635-server.virtnet: executing set_default_debug -1 all [10491.391355] Lustre: DEBUG MARKER: Using TIMEOUT=20 [10495.036271] Lustre: 343375:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify general/lod.*.mdt_hash=crush (mode = 0) failed: rc = -17 [10501.100652] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [10501.117346] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [10501.138735] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [10501.144020] Lustre: Skipped 3 previous similar messages [10505.147484] Lustre: server umount lustre-MDT0000 complete [10506.730310] LustreError: 340361:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [10506.747228] LustreError: 340361:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 3 previous similar messages [10508.494124] LustreError: 341506:0:(ldlm_lockd.c:2580:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1787200782 with bad export cookie 14374276770429829230 [10508.496693] LustreError: MGC192.168.206.135@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [10508.507444] LustreError: 341506:0:(ldlm_lockd.c:2580:ldlm_cancel_handler()) Skipped 4 previous similar messages [10508.727540] Lustre: server umount lustre-MDT0001 complete [10523.087968] Lustre: server umount lustre-OST0000 complete [10537.293582] Lustre: server umount lustre-OST0001 complete [10541.284886] Lustre: DEBUG MARKER: SKIP: conf-sanity test_114 skipping SLOW test 114 [10542.940960] Lustre: DEBUG MARKER: SKIP: conf-sanity test_115 skipping excluded test 115 [10544.898197] Lustre: DEBUG MARKER: == conf-sanity test 116: big size MDT support ============ 00:40:16 (1787200816) [10571.264844] Lustre: DEBUG MARKER: oleg635-server.virtnet: executing load_modules_local [10581.309977] loop0: detected capacity change from 0 to 1099511627776 [10582.067519] SGI XFS with ACLs, security attributes, quota, no debug enabled [10582.076750] *** VALIDATE xfs *** [10582.091932] XFS (loop0): Mounting V5 Filesystem [10582.117963] XFS (loop0): Ending clean mount [10583.308719] loop1: detected capacity change from 0 to 18691697672192 [10687.063883] Lustre: DEBUG MARKER: oleg635-server.virtnet: executing set_hostid [10697.936381] Lustre: DEBUG MARKER: oleg635-server.virtnet: executing load_modules_local [10731.671561] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [10731.858416] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [10731.891204] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [10731.896930] Lustre: lustre-MDT0000: new disk, initializing [10731.989204] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [10732.010171] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [10736.760945] Lustre: DEBUG MARKER: oleg635-server.virtnet: executing set_default_debug -1 all [10747.145166] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [10747.259150] Lustre: 348514:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [10747.301508] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [10747.303751] Lustre: Skipped 1 previous similar message [10747.307055] Lustre: lustre-MDT0001: new disk, initializing [10747.390393] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [10747.399565] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [10751.230554] Lustre: DEBUG MARKER: oleg635-server.virtnet: executing set_default_debug -1 all [10759.929425] Lustre: DEBUG MARKER: oleg635-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [10764.854842] Lustre: DEBUG MARKER: oleg635-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [10770.959252] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [10771.176145] Lustre: lustre-OST0000: new disk, initializing [10771.179831] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [10771.184659] Lustre: 350480:0:(osd_compat.c:1353:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [10772.715126] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [10772.722923] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [10772.826637] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [10776.283867] Lustre: DEBUG MARKER: oleg635-server.virtnet: executing set_default_debug -1 all [10784.341227] Lustre: DEBUG MARKER: oleg635-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [10788.022935] Lustre: DEBUG MARKER: oleg635-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 [10788.163632] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [10792.119665] Lustre: DEBUG MARKER: oleg635-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid 50 [10792.314405] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [10798.564127] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [10798.564303] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [10798.573589] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [10798.573596] Lustre: Skipped 3 previous similar messages [10798.606260] Lustre: Skipped 4 previous similar messages [10803.684359] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [10803.688994] Lustre: Skipped 2 previous similar messages [10808.287310] Lustre: lustre-OST0000 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [10808.412632] Lustre: server umount lustre-OST0000 complete [10813.921706] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [10813.931170] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [10813.935188] Lustre: Skipped 1 previous similar message [10817.276885] Lustre: server umount lustre-MDT0000 complete [10819.041714] LustreError: 348524:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [10819.054657] LustreError: 348524:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 1 previous similar message [10820.582894] LustreError: 348504:0:(ldlm_lockd.c:2580:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1787201094 with bad export cookie 14374276770429830777 [10820.585193] LustreError: MGC192.168.206.135@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [10820.596554] LustreError: 348504:0:(ldlm_lockd.c:2580:ldlm_cancel_handler()) Skipped 3 previous similar messages [10820.740443] Lustre: server umount lustre-MDT0001 complete [10826.642700] XFS (loop0): Unmounting Filesystem [10830.572843] Lustre: DEBUG MARKER: == conf-sanity test 117: lctl get_param return errors properly ========================================================== 00:45:02 (1787201102) [10836.519468] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [10836.831513] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [10836.835560] Lustre: Skipped 2 previous similar messages [10840.328591] Lustre: DEBUG MARKER: oleg635-server.virtnet: executing set_default_debug -1 all [10847.743297] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [10852.473763] Lustre: DEBUG MARKER: oleg635-server.virtnet: executing set_default_debug -1 all [10859.501956] Lustre: DEBUG MARKER: oleg635-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [10865.183312] Lustre: DEBUG MARKER: oleg635-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [10871.692458] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [10878.899196] Lustre: DEBUG MARKER: oleg635-server.virtnet: executing set_default_debug -1 all [10885.467378] Lustre: DEBUG MARKER: oleg635-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [10893.792842] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [10893.798536] LustreError: Skipped 1 previous similar message [10893.801196] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [10893.811746] Lustre: Skipped 1 previous similar message [10893.822222] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [10897.896822] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [10907.108336] Lustre: lustre-OST0000 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [10907.236116] Lustre: server umount lustre-OST0000 complete [10916.773158] Lustre: server umount lustre-MDT0000 complete [10918.369611] LustreError: 353040:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [10918.381157] LustreError: 353040:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 4 previous similar messages [10919.546776] LustreError: 355918:0:(ldlm_lockd.c:2580:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1787201193 with bad export cookie 14374276770429831666 [10919.548981] LustreError: MGC192.168.206.135@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [10919.559327] LustreError: 355918:0:(ldlm_lockd.c:2580:ldlm_cancel_handler()) Skipped 3 previous similar messages [10919.711592] Lustre: server umount lustre-MDT0001 complete [10929.112363] Lustre: DEBUG MARKER: oleg635-server.virtnet: executing unload_modules_local [10931.691919] Key type lgssc unregistered [10932.107525] LNet: 356673:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [10932.124200] LNetError: 356673:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [10932.146458] LNet: Removed LNI 192.168.206.135@tcp [10932.960695] Key type .llcrypt unregistered [10932.965546] Key type ._llcrypt unregistered [10960.035435] Key type ._llcrypt registered [10960.037355] Key type .llcrypt registered [10960.156459] Lustre: DEBUG MARKER: oleg635-server.virtnet: executing load_modules_local [10961.134803] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [10961.183106] alg: No test for adler32 (adler32-zlib) [10962.220489] Lustre: Lustre: Build Version: 2.17.57_44_gafaaa93 [10962.460865] LNet: Added LNI 192.168.206.135@tcp [8/256/0/180] [10964.129143] Key type lgssc registered [10965.316433] Lustre: Echo OBD driver; http://www.lustre.org/ [10971.686234] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [10971.701932] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [10972.941509] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [10976.396709] Lustre: DEBUG MARKER: oleg635-server.virtnet: executing set_default_debug -1 all [10982.375088] LustreError: 358242:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [10982.395043] LustreError: 358242:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 2 previous similar messages [10983.565214] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [10983.791051] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [10987.352772] Lustre: DEBUG MARKER: oleg635-server.virtnet: executing set_default_debug -1 all [10993.378731] Lustre: DEBUG MARKER: oleg635-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [10999.129873] Lustre: DEBUG MARKER: oleg635-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [11005.174223] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [11005.507629] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [11011.187806] Lustre: DEBUG MARKER: oleg635-server.virtnet: executing set_default_debug -1 all [11017.935962] Lustre: DEBUG MARKER: oleg635-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [11024.864960] LustreError: lustre-MDT0001-osp-MDT0000: operation mds_statfs to node 0@lo failed: rc = -107 [11024.877723] Lustre: lustre-MDT0001-osp-MDT0000: Connection to lustre-MDT0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [11024.898412] Lustre: lustre-MDT0001: Not available for connect from 0@lo (stopping) [11028.448405] Lustre: lustre-MDT0001: Not available for connect from 0@lo (stopping) [11028.454886] Lustre: lustre-MDT0001-lwp-OST0000: Connection to lustre-MDT0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [11028.459083] Lustre: Skipped 1 previous similar message [11030.047269] Lustre: lustre-MDT0001: Not available for connect from 192.168.206.35@tcp (stopping) [11030.054326] Lustre: Skipped 1 previous similar message [11030.281604] Lustre: server umount lustre-MDT0001 complete [11033.572583] LustreError: 359337:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [11033.586942] LustreError: 359337:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 1 previous similar message [11035.174238] LustreError: 358237:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 192.168.206.35@tcp (no target). If you are running an HA pair check that the target is mounted on the other server. [11038.705169] LustreError: 358241:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [11038.734852] LustreError: 358241:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 1 previous similar message [11040.027541] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [11040.166484] Lustre: MGS: Regenerating lustre-MDT0001 log by user request: rc = 0 [11040.171425] Lustre: Found index 1 for lustre-MDT0001, updating log [11040.211502] Lustre: 358228:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [11040.469072] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [11044.309116] Lustre: DEBUG MARKER: oleg635-server.virtnet: executing set_default_debug -1 all [11045.860040] Lustre: lustre-MDT0001-lwp-OST0000: Connection restored to 0@lo (at 0@lo) [11049.465291] LustreError: 361842:0:(obd_mount.c:234:lustre_start_simple()) lustre-MDT0001-lwp-OST0000 attach error -17 [11049.474875] LustreError: 361842:0:(tgt_mount.c:675:lustre_lwp_setup()) lustre-MDT0001-lwp-OST0000: setup up failed: rc -17 [11049.484166] LustreError: 358233:0:(mgc_request.c:614:do_requeue()) failed processing log: -17 [11049.567406] LustreError: 361845:0:(obd_config.c:2102:class_config_llog_handler()) MGC192.168.206.135@tcp: cfg command failed: rc = -17 [11049.581919] Lustre: cmd=cf001 0:lustre-MDT0001-osp-MDT0000 1:osp 2:lustre-MDT0000-mdtlov_UUID [11049.581919] [11352.638944] Lustre: lustre-MDT0001: Not available for connect from 192.168.206.35@tcp (stopping) [11353.056896] LustreError: lustre-MDT0001-osp-MDT0000: operation mds_statfs to node 0@lo failed: rc = -107 [11353.070055] Lustre: lustre-MDT0001-osp-MDT0000: Connection to lustre-MDT0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [11353.092163] Lustre: Skipped 1 previous similar message [11355.835093] Lustre: server umount lustre-MDT0001 complete [11357.729065] LustreError: 360557:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 192.168.206.35@tcp (no target). If you are running an HA pair check that the target is mounted on the other server. [11357.752518] LustreError: 360557:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 1 previous similar message [11365.583571] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [11365.658437] Lustre: MGS: Regenerating lustre-MDT0001 log by user request: rc = 0 [11365.663718] Lustre: Found index 1 for lustre-MDT0001, updating log [11365.702773] Lustre: 358229:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [11365.901664] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [11369.566326] Lustre: DEBUG MARKER: oleg635-server.virtnet: executing set_default_debug -1 all [11370.997829] Lustre: lustre-MDT0001-lwp-OST0000: Connection restored to 0@lo (at 0@lo) [11374.580821] LustreError: 370984:0:(obd_mount.c:234:lustre_start_simple()) lustre-MDT0001-lwp-OST0000 attach error -17 [11374.592538] LustreError: 370984:0:(tgt_mount.c:675:lustre_lwp_setup()) lustre-MDT0001-lwp-OST0000: setup up failed: rc -17 [11374.605844] LustreError: 358233:0:(mgc_request.c:614:do_requeue()) failed processing log: -17 [11374.611427] LustreError: 358233:0:(mgc_request.c:614:do_requeue()) Skipped 1 previous similar message [11374.678649] LustreError: 370988:0:(obd_config.c:761:class_setup()) osp: device 8 already setup: rc = -17 [11374.684318] LustreError: 370988:0:(obd_config.c:866:class_detach()) OBD device 8 still set up [11374.690334] LustreError: 370988:0:(obd_config.c:2102:class_config_llog_handler()) MGC192.168.206.135@tcp: cfg command failed: rc = -17 [11374.699944] Lustre: cmd=cf003 0:lustre-MDT0001-osp-MDT0000 1:lustre-MDT0001_UUID 2:192.168.206.135@tcp [11374.699944] [11678.185235] Lustre: lustre-MDT0001-osp-MDT0000: Connection to lustre-MDT0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [11678.185544] LustreError: lustre-MDT0001-osp-MDT0000: operation mds_statfs to node 0@lo failed: rc = -107 [11678.186481] Lustre: lustre-MDT0001: Not available for connect from 0@lo (stopping) [11678.186487] Lustre: Skipped 2 previous similar messages [11678.194843] Lustre: Skipped 1 previous similar message [11681.608446] Lustre: server umount lustre-MDT0001 complete [11683.299991] LustreError: 358241:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [11683.323722] LustreError: 358241:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 6 previous similar messages [11690.958727] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [11691.041519] Lustre: MGS: Regenerating lustre-MDT0001 log by user request: rc = 0 [11691.045671] Lustre: Found index 1 for lustre-MDT0001, updating log [11691.076212] Lustre: 358229:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [11691.286845] Lustre: lustre-MDT0001: Imperative Recovery enabled, recovery window shrunk from 60-180 down to 60-180 [11695.383392] Lustre: DEBUG MARKER: oleg635-server.virtnet: executing set_default_debug -1 all [11696.617466] Lustre: lustre-MDT0001-lwp-OST0000: Connection restored to 0@lo (at 0@lo) [11700.214069] LustreError: 379848:0:(obd_mount.c:234:lustre_start_simple()) lustre-MDT0001-lwp-OST0000 attach error -17 [11700.224371] LustreError: 379848:0:(tgt_mount.c:675:lustre_lwp_setup()) lustre-MDT0001-lwp-OST0000: setup up failed: rc -17 [11700.234552] LustreError: 358233:0:(mgc_request.c:614:do_requeue()) failed processing log: -17 [11700.239903] LustreError: 358233:0:(mgc_request.c:614:do_requeue()) Skipped 1 previous similar message [11700.311055] LustreError: 379852:0:(lod_lov.c:208:lod_add_device()) lustre-MDT0001-osp-MDT0000: cannot connect to next dev lustre-MDT0001_UUID (-114) [11700.325875] LustreError: 379852:0:(obd_config.c:2102:class_config_llog_handler()) MGC192.168.206.135@tcp: cfg command failed: rc = -114 [11700.334276] Lustre: cmd=cf014 0:lustre-MDT0000-mdtlov 1:lustre-MDT0001_UUID 2:1 3:1 [11700.334276] [12019.173723] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [12019.175957] Lustre: lustre-MDT0000-lwp-OST0000: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [12019.196817] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [12019.215453] Lustre: Skipped 1 previous similar message [12019.227472] Lustre: Skipped 4 previous similar messages [12023.317638] Lustre: server umount lustre-MDT0000 complete [12024.292371] LustreError: 360557:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [12024.315976] LustreError: 360557:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 6 previous similar messages [12026.887906] LustreError: 358222:0:(ldlm_lockd.c:2580:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1787202300 with bad export cookie 10240289489832770726 [12026.892371] LustreError: MGC192.168.206.135@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [12026.896605] LustreError: 358222:0:(ldlm_lockd.c:2580:ldlm_cancel_handler()) Skipped 4 previous similar messages [12029.417750] Lustre: lustre-MDT0001-lwp-OST0000: Connection to lustre-MDT0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [12029.437041] Lustre: Skipped 1 previous similar message [12036.589022] Lustre: lustre-MDT0001: Not available for connect from 0@lo (stopping) [12036.597705] Lustre: Skipped 2 previous similar messages [12041.183172] Lustre: lustre-MDT0001 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [12041.377777] Lustre: server umount lustre-MDT0001 complete [12045.299398] Lustre: server umount lustre-OST0000 complete [12054.522758] Lustre: DEBUG MARKER: oleg635-server.virtnet: executing set_hostid [12063.486563] Lustre: DEBUG MARKER: oleg635-server.virtnet: executing load_modules_local [12091.858861] Lustre: DEBUG MARKER: == conf-sanity test 120: cross-target rename should not create bad symlinks ========================================================== 01:06:03 (1787202363) [12098.272189] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [12098.505824] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [12098.564346] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [12098.580735] Lustre: lustre-MDT0000: new disk, initializing [12098.680054] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [12098.702619] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [12102.595095] Lustre: DEBUG MARKER: oleg635-server.virtnet: executing set_default_debug -1 all [12111.855580] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [12111.959763] Lustre: 391280:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [12112.019465] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [12112.029360] Lustre: Skipped 1 previous similar message [12112.034298] Lustre: lustre-MDT0001: new disk, initializing [12112.122174] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [12112.161590] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [12112.172058] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [12116.142910] Lustre: DEBUG MARKER: oleg635-server.virtnet: executing set_default_debug -1 all [12125.297451] Lustre: DEBUG MARKER: oleg635-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [12130.565015] Lustre: DEBUG MARKER: oleg635-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [12136.541528] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [12136.932601] Lustre: lustre-OST0000: new disk, initializing [12136.939386] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [12136.958108] Lustre: 393247:0:(osd_compat.c:1353:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [12137.054742] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [12142.917788] Lustre: DEBUG MARKER: oleg635-server.virtnet: executing set_default_debug -1 all [12146.246059] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [12146.259331] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [12146.404593] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [12151.169887] Lustre: DEBUG MARKER: oleg635-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [12158.432326] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [12158.448345] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [12158.459963] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [12163.959290] Lustre: server umount lustre-MDT0000 complete [12167.137462] LustreError: 391293:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [12167.154676] LustreError: 391293:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 5 previous similar messages [12167.507933] LustreError: MGC192.168.206.135@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [12167.507418] LustreError: 392075:0:(ldlm_lockd.c:2580:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1787202441 with bad export cookie 10240289489832772672 [12167.534309] LustreError: 392087:0:(osp_object.c:618:osp_attr_get()) lustre-MDT0000-osp-MDT0001: osp_attr_get update error [0x200000401:0x2:0x0]: rc = -5 [12167.537491] LustreError: 392075:0:(ldlm_lockd.c:2580:ldlm_cancel_handler()) Skipped 1 previous similar message [12167.708268] Lustre: server umount lustre-MDT0001 complete [12182.122616] Lustre: server umount lustre-OST0000 complete [12211.930173] Lustre: DEBUG MARKER: oleg635-server.virtnet: executing set_hostid [12221.151633] Lustre: DEBUG MARKER: oleg635-server.virtnet: executing load_modules_local [12255.814430] Lustre: DEBUG MARKER: == conf-sanity test complete, duration 12016 sec ========= 01:08:47 (1787202527) [12257.176277] Lustre: DEBUG MARKER: === conf-sanity: start cleanup 01:08:49 (1787202529) === [12260.615256] Lustre: DEBUG MARKER: === conf-sanity: finish cleanup 01:08:51 (1787202531) === [12301.068373] Lustre: DEBUG MARKER: oleg635-server.virtnet: executing unload_modules_local [12303.761342] Key type lgssc unregistered [12304.090589] LNet: 399618:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [12304.095985] LNetError: 399618:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [12304.107339] LNet: Removed LNI 192.168.206.135@tcp [12304.908611] Key type .llcrypt unregistered [12304.911425] Key type ._llcrypt unregistered