[ 0.000000] Linux version 4.18.0rh8.10-debug (green@maintenance) (gcc version 8.5.0 20210514 (Red Hat 8.5.0-26) (GCC)) #2 SMP Mon Jul 14 01:24:22 EDT 2025 [ 0.000000] Command line: rd.shell root=nbd:192.168.200.253:rocky8.10:ext4:ro:-p,-b4096 ro crashkernel=256M panic=1 nomodeset ipmtu=9000 ip=dhcp rd.neednet=1 init_on_free=off mitigations=off console=ttyS1,115200 audit=0 [ 0.000000] x86/fpu: Supporting XSAVE feature 0x001: 'x87 floating point registers' [ 0.000000] x86/fpu: Supporting XSAVE feature 0x002: 'SSE registers' [ 0.000000] x86/fpu: Supporting XSAVE feature 0x004: 'AVX registers' [ 0.000000] x86/fpu: xstate_offset[2]: 576, xstate_sizes[2]: 256 [ 0.000000] x86/fpu: Enabled xstate features 0x7, context size is 832 bytes, using 'standard' format. [ 0.000000] signal: max sigframe size: 1776 [ 0.000000] BIOS-provided physical RAM map: [ 0.000000] BIOS-e820: [mem 0x0000000000000000-0x000000000009fbff] usable [ 0.000000] BIOS-e820: [mem 0x000000000009fc00-0x000000000009ffff] reserved [ 0.000000] BIOS-e820: [mem 0x00000000000f0000-0x00000000000fffff] reserved [ 0.000000] BIOS-e820: [mem 0x0000000000100000-0x00000000bffcdfff] usable [ 0.000000] BIOS-e820: [mem 0x00000000bffce000-0x00000000bfffffff] reserved [ 0.000000] BIOS-e820: [mem 0x00000000feffc000-0x00000000feffffff] reserved [ 0.000000] BIOS-e820: [mem 0x00000000fffc0000-0x00000000ffffffff] reserved [ 0.000000] BIOS-e820: [mem 0x0000000100000000-0x0000000146dfffff] usable [ 0.000000] NX (Execute Disable) protection: active [ 0.000000] SMBIOS 2.8 present. [ 0.000000] DMI: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.17.0-10.fc44 06/10/2025 [ 0.000000] Hypervisor detected: KVM [ 0.000000] kvm-clock: Using msrs 4b564d01 and 4b564d00 [ 0.000000] kvm-clock: using sched offset of 553754411 cycles [ 0.000000] clocksource: kvm-clock: mask: 0xffffffffffffffff max_cycles: 0x1cd42e4dffb, max_idle_ns: 881590591483 ns [ 0.000000] tsc: Detected 2399.998 MHz processor [ 0.000000] last_pfn = 0x146e00 max_arch_pfn = 0x400000000 [ 0.000000] x86/PAT: Configuration [0-7]: WB WC UC- UC WB WP UC- WT [ 0.000000] last_pfn = 0xbffce max_arch_pfn = 0x400000000 [ 0.000000] found SMP MP-table at [mem 0x000f54b0-0x000f54bf] [ 0.000000] RAMDISK: [mem 0xbcc54000-0xbffbffff] [ 0.000000] ACPI: Early table checksum verification disabled [ 0.000000] ACPI: RSDP 0x00000000000F52D0 000014 (v00 BOCHS ) [ 0.000000] ACPI: RSDT 0x00000000BFFE247C 000034 (v01 BOCHS BXPC 00000001 BXPC 00000001) [ 0.000000] ACPI: FACP 0x00000000BFFE2318 000074 (v01 BOCHS BXPC 00000001 BXPC 00000001) [ 0.000000] ACPI: DSDT 0x00000000BFFE0040 0022D8 (v01 BOCHS BXPC 00000001 BXPC 00000001) [ 0.000000] ACPI: FACS 0x00000000BFFE0000 000040 [ 0.000000] ACPI: APIC 0x00000000BFFE238C 000090 (v03 BOCHS BXPC 00000001 BXPC 00000001) [ 0.000000] ACPI: HPET 0x00000000BFFE241C 000038 (v01 BOCHS BXPC 00000001 BXPC 00000001) [ 0.000000] ACPI: WAET 0x00000000BFFE2454 000028 (v01 BOCHS BXPC 00000001 BXPC 00000001) [ 0.000000] ACPI: Reserving FACP table memory at [mem 0xbffe2318-0xbffe238b] [ 0.000000] ACPI: Reserving DSDT table memory at [mem 0xbffe0040-0xbffe2317] [ 0.000000] ACPI: Reserving FACS table memory at [mem 0xbffe0000-0xbffe003f] [ 0.000000] ACPI: Reserving APIC table memory at [mem 0xbffe238c-0xbffe241b] [ 0.000000] ACPI: Reserving HPET table memory at [mem 0xbffe241c-0xbffe2453] [ 0.000000] ACPI: Reserving WAET table memory at [mem 0xbffe2454-0xbffe247b] [ 0.000000] No NUMA configuration found [ 0.000000] Faking a node at [mem 0x0000000000000000-0x0000000146dfffff] [ 0.000000] NODE_DATA(0) allocated [mem 0x1465a3000-0x1465cdfff] [ 0.000000] Reserving 256MB of memory at 2752MB for crashkernel (System RAM: 4205MB) [ 0.000000] Zone ranges: [ 0.000000] DMA [mem 0x0000000000001000-0x0000000000ffffff] [ 0.000000] DMA32 [mem 0x0000000001000000-0x00000000ffffffff] [ 0.000000] Normal [mem 0x0000000100000000-0x0000000146dfffff] [ 0.000000] Device empty [ 0.000000] Movable zone start for each node [ 0.000000] Early memory node ranges [ 0.000000] node 0: [mem 0x0000000000001000-0x000000000009efff] [ 0.000000] node 0: [mem 0x0000000000100000-0x00000000bffcdfff] [ 0.000000] node 0: [mem 0x0000000100000000-0x0000000146dfffff] [ 0.000000] Zeroed struct page in unavailable ranges: 4756 pages [ 0.000000] Initmem setup node 0 [mem 0x0000000000001000-0x0000000146dfffff] [ 0.000000] ACPI: PM-Timer IO Port: 0x608 [ 0.000000] ACPI: LAPIC_NMI (acpi_id[0xff] dfl dfl lint[0x1]) [ 0.000000] IOAPIC[0]: apic_id 0, version 17, address 0xfec00000, GSI 0-23 [ 0.000000] ACPI: INT_SRC_OVR (bus 0 bus_irq 0 global_irq 2 dfl dfl) [ 0.000000] ACPI: INT_SRC_OVR (bus 0 bus_irq 5 global_irq 5 high level) [ 0.000000] ACPI: INT_SRC_OVR (bus 0 bus_irq 9 global_irq 9 high level) [ 0.000000] ACPI: INT_SRC_OVR (bus 0 bus_irq 10 global_irq 10 high level) [ 0.000000] ACPI: INT_SRC_OVR (bus 0 bus_irq 11 global_irq 11 high level) [ 0.000000] Using ACPI (MADT) for SMP configuration information [ 0.000000] ACPI: HPET id: 0x8086a201 base: 0xfed00000 [ 0.000000] TSC deadline timer available [ 0.000000] smpboot: Allowing 4 CPUs, 0 hotplug CPUs [ 0.000000] kvm-guest: KVM setup pv remote TLB flush [ 0.000000] kvm-guest: setup PV sched yield [ 0.000000] PM: Registered nosave memory: [mem 0x00000000-0x00000fff] [ 0.000000] PM: Registered nosave memory: [mem 0x0009f000-0x0009ffff] [ 0.000000] PM: Registered nosave memory: [mem 0x000a0000-0x000effff] [ 0.000000] PM: Registered nosave memory: [mem 0x000f0000-0x000fffff] [ 0.000000] PM: Registered nosave memory: [mem 0xbffce000-0xbfffffff] [ 0.000000] PM: Registered nosave memory: [mem 0xc0000000-0xfeffbfff] [ 0.000000] PM: Registered nosave memory: [mem 0xfeffc000-0xfeffffff] [ 0.000000] PM: Registered nosave memory: [mem 0xff000000-0xfffbffff] [ 0.000000] PM: Registered nosave memory: [mem 0xfffc0000-0xffffffff] [ 0.000000] [mem 0xc0000000-0xfeffbfff] available for PCI devices [ 0.000000] Booting paravirtualized kernel on KVM [ 0.000000] clocksource: refined-jiffies: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 1910969940391419 ns [ 0.000000] setup_percpu: NR_CPUS:8192 nr_cpumask_bits:4 nr_cpu_ids:4 nr_node_ids:1 [ 0.000000] percpu: Embedded 63 pages/cpu s221184 r8192 d28672 u524288 [ 0.000000] kvm-guest: PV spinlocks enabled [ 0.000000] PV qspinlock hash table entries: 256 (order: 0, 4096 bytes, linear) [ 0.000000] Built 1 zonelists, mobility grouping on. Total pages: 1059606 [ 0.000000] Policy zone: Normal [ 0.000000] Kernel command line: rd.shell root=nbd:192.168.200.253:rocky8.10:ext4:ro:-p,-b4096 ro crashkernel=256M panic=1 nomodeset ipmtu=9000 ip=dhcp rd.neednet=1 init_on_free=off mitigations=off console=ttyS1,115200 audit=0 [ 0.000000] Specific versions of hardware are certified with Red Hat Enterprise Linux 8. Please see the list of hardware certified with Red Hat Enterprise Linux 8 at https://catalog.redhat.com. [ 0.000000] audit: disabled (until reboot) [ 0.000000] software IO TLB: area num 4. [ 0.000000] Memory: 2829652K/4306352K available (18435K kernel code, 11221K rwdata, 7248K rodata, 2908K init, 18040K bss, 524584K reserved, 0K cma-reserved) [ 0.000000] SLUB: HWalign=64, Order=0-3, MinObjects=0, CPUs=4, Nodes=1 [ 0.000000] kmemleak: Kernel memory leak detector disabled [ 0.000000] ftrace: allocating 41240 entries in 162 pages [ 0.000000] ftrace: allocated 162 pages with 3 groups [ 0.000000] rcu: Hierarchical RCU implementation. [ 0.000000] rcu: RCU event tracing is enabled. [ 0.000000] rcu: RCU restricting CPUs from NR_CPUS=8192 to nr_cpu_ids=4. [ 0.000000] rcu: RCU callback double-/use-after-free debug enabled. [ 0.000000] Rude variant of Tasks RCU enabled. [ 0.000000] Tracing variant of Tasks RCU enabled. [ 0.000000] rcu: RCU calculated value of scheduler-enlistment delay is 100 jiffies. [ 0.000000] rcu: Adjusting geometry for rcu_fanout_leaf=16, nr_cpu_ids=4 [ 0.000000] NR_IRQS: 524544, nr_irqs: 456, preallocated irqs: 16 [ 0.000000] random: get_random_bytes called from start_kernel+0x622/0x9a8 with crng_init=0 [ 0.001000] Console: colour *CGA 80x25 [ 0.001000] printk: console [ttyS1] enabled [ 0.001000] ACPI: Core revision 20220331 [ 0.001000] clocksource: hpet: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 19112604467 ns [ 0.001013] APIC: Switch to symmetric I/O mode setup [ 0.003288] x2apic enabled [ 0.004007] Switched APIC routing to physical x2apic. [ 0.005012] kvm-guest: setup PV IPIs [ 0.008000] ..TIMER: vector=0x30 apic1=0 pin1=2 apic2=-1 pin2=-1 [ 0.008000] clocksource: tsc-early: mask: 0xffffffffffffffff max_cycles: 0x229835b7123, max_idle_ns: 440795242976 ns [ 0.008031] Calibrating delay loop (skipped) preset value.. 4799.99 BogoMIPS (lpj=2399998) [ 0.009018] pid_max: default: 32768 minimum: 301 [ 0.010214] LSM: Security Framework initializing [ 0.011078] Yama: becoming mindful. [ 0.012000] SELinux: Initializing. [ 0.012113] *** VALIDATE selinux *** [ 0.020804] Dentry cache hash table entries: 1048576 (order: 11, 8388608 bytes, vmalloc) [ 0.025968] Inode-cache hash table entries: 524288 (order: 10, 4194304 bytes, vmalloc) [ 0.026206] Mount-cache hash table entries: 16384 (order: 5, 131072 bytes, vmalloc) [ 0.027151] Mountpoint-cache hash table entries: 16384 (order: 5, 131072 bytes, vmalloc) [ 0.028156] *** VALIDATE tmpfs *** [ 0.030459] *** VALIDATE proc *** [ 0.032011] *** VALIDATE cgroup *** [ 0.033007] *** VALIDATE cgroup2 *** [ 0.034259] x86/cpu: User Mode Instruction Prevention (UMIP) activated [ 0.036091] Last level iTLB entries: 4KB 0, 2MB 0, 4MB 0 [ 0.037007] Last level dTLB entries: 4KB 0, 2MB 0, 4MB 0, 1GB 0 [ 0.038025] Spectre V2 : User space: Vulnerable [ 0.039008] Speculative Store Bypass: Vulnerable [ 0.041734] debug: unmapping init [mem 0xffffffffb8059000-0xffffffffb8060fff] [ 0.043266] smpboot: CPU0: Intel(R) Xeon(R) CPU E5-2695 v2 @ 2.40GHz (family: 0x6, model: 0x3e, stepping: 0x4) [ 0.045466] Performance Events: IvyBridge events, full-width counters, Intel PMU driver. [ 0.046025] ... version: 2 [ 0.047010] ... bit width: 48 [ 0.048013] ... generic registers: 4 [ 0.049011] ... value mask: 0000ffffffffffff [ 0.050011] ... max period: 00007fffffffffff [ 0.051009] ... fixed-purpose events: 3 [ 0.052012] ... event mask: 000000070000000f [ 0.053363] rcu: Hierarchical SRCU implementation. [ 0.055946] smp: Bringing up secondary CPUs ... [ 0.056660] x86: Booting SMP configuration: [ 0.057028] .... node #0, CPUs: #1 #2 #3 [ 0.064182] smp: Brought up 1 node, 4 CPUs [ 0.066018] smpboot: Max logical packages: 1 [ 0.067010] smpboot: Total of 4 processors activated (19199.98 BogoMIPS) [ 0.088053] node 0 deferred pages initialised in 18ms [ 0.091270] devtmpfs: initialized [ 0.092216] x86/mm: Memory block size: 128MB [ 0.095000] gcov: version magic: 0x41383552 [ 0.097195] clocksource: jiffies: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 1911260446275000 ns [ 0.100117] futex hash table entries: 1024 (order: 4, 65536 bytes, vmalloc) [ 0.101326] pinctrl core: initialized pinctrl subsystem [ 0.102199] [ 0.102598] ************************************************************* [ 0.105010] ** NOTICE NOTICE NOTICE NOTICE NOTICE NOTICE NOTICE ** [ 0.106006] ** ** [ 0.108008] ** IOMMU DebugFS SUPPORT HAS BEEN ENABLED IN THIS KERNEL ** [ 0.109009] ** ** [ 0.111011] ** This means that this kernel is built to expose internal ** [ 0.112009] ** IOMMU data structures, which may compromise security on ** [ 0.114012] ** your system. ** [ 0.117015] ** ** [ 0.120012] ** If you see this message and you are not debugging the ** [ 0.123014] ** kernel, report this immediately to your vendor! ** [ 0.127014] ** ** [ 0.129010] ** NOTICE NOTICE NOTICE NOTICE NOTICE NOTICE NOTICE ** [ 0.131011] ************************************************************* [ 0.134704] NET: Registered protocol family 16 [ 0.136463] DMA: preallocated 512 KiB GFP_KERNEL pool for atomic allocations [ 0.139056] DMA: preallocated 512 KiB GFP_KERNEL|GFP_DMA pool for atomic allocations [ 0.142056] DMA: preallocated 512 KiB GFP_KERNEL|GFP_DMA32 pool for atomic allocations [ 0.146172] cpuidle: using governor menu [ 0.147879] acpiphp: ACPI Hot Plug PCI Controller Driver version: 0.5 [ 0.150569] PCI: Using configuration type 1 for base access [ 0.153197] core: PMU erratum BJ122, BV98, HSD29 worked around, HT is on [ 0.263251] HugeTLB registered 1.00 GiB page size, pre-allocated 0 pages [ 0.264078] HugeTLB registered 2.00 MiB page size, pre-allocated 0 pages [ 0.266083] cryptd: max_cpu_qlen set to 1000 [ 0.270453] ACPI: Added _OSI(Module Device) [ 0.271017] ACPI: Added _OSI(Processor Device) [ 0.274016] ACPI: Added _OSI(3.0 _SCP Extensions) [ 0.276027] ACPI: Added _OSI(Processor Aggregator Device) [ 0.283632] ACPI: 1 ACPI AML tables successfully acquired and loaded [ 0.292454] ACPI: Interpreter enabled [ 0.295076] ACPI: PM: (supports S0 S3 S4 S5) [ 0.298014] ACPI: Using IOAPIC for interrupt routing [ 0.301912] PCI: Using host bridge windows from ACPI; if necessary, use "pci=nocrs" and report a bug [ 0.306660] ACPI: Enabled 2 GPEs in block 00 to 0F [ 0.321189] ACPI: PCI Root Bridge [PCI0] (domain 0000 [bus 00-ff]) [ 0.323052] acpi PNP0A03:00: _OSC: OS supports [ASPM ClockPM Segments MSI HPX-Type3] [ 0.326021] acpi PNP0A03:00: _OSC: not requesting OS control; OS requires [ExtendedConfig ASPM ClockPM MSI] [ 0.329121] acpi PNP0A03:00: fail to add MMCONFIG information, can't access extended PCI configuration space under this bridge. [ 0.334856] acpiphp: Slot [2] registered [ 0.336114] acpiphp: Slot [5] registered [ 0.338173] acpiphp: Slot [6] registered [ 0.340208] acpiphp: Slot [7] registered [ 0.341114] acpiphp: Slot [8] registered [ 0.343130] acpiphp: Slot [9] registered [ 0.345143] acpiphp: Slot [10] registered [ 0.347119] acpiphp: Slot [3] registered [ 0.348284] acpiphp: Slot [4] registered [ 0.350238] acpiphp: Slot [11] registered [ 0.352125] acpiphp: Slot [12] registered [ 0.354251] acpiphp: Slot [13] registered [ 0.355437] acpiphp: Slot [14] registered [ 0.357405] acpiphp: Slot [15] registered [ 0.360171] acpiphp: Slot [16] registered [ 0.361141] acpiphp: Slot [17] registered [ 0.363122] acpiphp: Slot [18] registered [ 0.364138] acpiphp: Slot [19] registered [ 0.366198] acpiphp: Slot [20] registered [ 0.368117] acpiphp: Slot [21] registered [ 0.369329] acpiphp: Slot [22] registered [ 0.371309] acpiphp: Slot [23] registered [ 0.373100] acpiphp: Slot [24] registered [ 0.375084] acpiphp: Slot [25] registered [ 0.376161] acpiphp: Slot [26] registered [ 0.378122] acpiphp: Slot [27] registered [ 0.380206] acpiphp: Slot [28] registered [ 0.382113] acpiphp: Slot [29] registered [ 0.383237] acpiphp: Slot [30] registered [ 0.385094] acpiphp: Slot [31] registered [ 0.386238] PCI host bridge to bus 0000:00 [ 0.388021] pci_bus 0000:00: root bus resource [io 0x0000-0x0cf7 window] [ 0.391027] pci_bus 0000:00: root bus resource [io 0x0d00-0xffff window] [ 0.394034] pci_bus 0000:00: root bus resource [mem 0x000a0000-0x000bffff window] [ 0.396019] pci_bus 0000:00: root bus resource [mem 0xc0000000-0xfebfffff window] [ 0.398019] pci_bus 0000:00: root bus resource [mem 0xe0000000000-0xe007fffffff window] [ 0.400023] pci_bus 0000:00: root bus resource [bus 00-ff] [ 0.402148] pci 0000:00:00.0: [8086:1237] type 00 class 0x060000 [ 0.403990] pci 0000:00:01.0: [8086:7000] type 00 class 0x060100 [ 0.407136] pci 0000:00:01.1: [8086:7010] type 00 class 0x010180 [ 0.420654] pci 0000:00:01.1: reg 0x20: [io 0xc320-0xc32f] [ 0.423729] pci 0000:00:01.1: legacy IDE quirk: reg 0x10: [io 0x01f0-0x01f7] [ 0.426013] pci 0000:00:01.1: legacy IDE quirk: reg 0x14: [io 0x03f6] [ 0.427010] pci 0000:00:01.1: legacy IDE quirk: reg 0x18: [io 0x0170-0x0177] [ 0.429013] pci 0000:00:01.1: legacy IDE quirk: reg 0x1c: [io 0x0376] [ 0.431312] pci 0000:00:01.3: [8086:7113] type 00 class 0x068000 [ 0.433571] pci 0000:00:01.3: quirk: [io 0x0600-0x063f] claimed by PIIX4 ACPI [ 0.435031] pci 0000:00:01.3: quirk: [io 0x0700-0x070f] claimed by PIIX4 SMB [ 0.437617] pci 0000:00:02.0: [1af4:1000] type 00 class 0x020000 [ 0.445029] pci 0000:00:02.0: reg 0x10: [io 0xc300-0xc31f] [ 0.459018] pci 0000:00:02.0: reg 0x20: [mem 0xe0000000000-0xe0000003fff 64bit pref] [ 0.465020] pci 0000:00:02.0: reg 0x30: [mem 0xfeb80000-0xfebbffff pref] [ 0.472062] pci 0000:00:05.0: [1af4:1001] type 00 class 0x010000 [ 0.490028] pci 0000:00:05.0: reg 0x10: [io 0xc000-0xc07f] [ 0.509027] pci 0000:00:05.0: reg 0x14: [mem 0xfebc0000-0xfebc0fff] [ 0.531021] pci 0000:00:05.0: reg 0x20: [mem 0xe0000004000-0xe0000007fff 64bit pref] [ 0.538000] pci 0000:00:06.0: [1af4:1001] type 00 class 0x010000 [ 0.544016] pci 0000:00:06.0: reg 0x10: [io 0xc080-0xc0ff] [ 0.573039] pci 0000:00:06.0: reg 0x14: [mem 0xfebc1000-0xfebc1fff] [ 0.608025] pci 0000:00:06.0: reg 0x20: [mem 0xe0000008000-0xe000000bfff 64bit pref] [ 0.623182] pci 0000:00:07.0: [1af4:1001] type 00 class 0x010000 [ 0.633023] pci 0000:00:07.0: reg 0x10: [io 0xc100-0xc17f] [ 0.642026] pci 0000:00:07.0: reg 0x14: [mem 0xfebc2000-0xfebc2fff] [ 0.668028] pci 0000:00:07.0: reg 0x20: [mem 0xe000000c000-0xe000000ffff 64bit pref] [ 0.682266] pci 0000:00:08.0: [1af4:1001] type 00 class 0x010000 [ 0.691023] pci 0000:00:08.0: reg 0x10: [io 0xc180-0xc1ff] [ 0.698021] pci 0000:00:08.0: reg 0x14: [mem 0xfebc3000-0xfebc3fff] [ 0.722026] pci 0000:00:08.0: reg 0x20: [mem 0xe0000010000-0xe0000013fff 64bit pref] [ 0.738000] pci 0000:00:09.0: [1af4:1001] type 00 class 0x010000 [ 0.765031] pci 0000:00:09.0: reg 0x10: [io 0xc200-0xc27f] [ 0.776022] pci 0000:00:09.0: reg 0x14: [mem 0xfebc4000-0xfebc4fff] [ 0.806024] pci 0000:00:09.0: reg 0x20: [mem 0xe0000014000-0xe0000017fff 64bit pref] [ 0.833634] pci 0000:00:0a.0: [1af4:1001] type 00 class 0x010000 [ 0.852022] pci 0000:00:0a.0: reg 0x10: [io 0xc280-0xc2ff] [ 0.915042] pci 0000:00:0a.0: reg 0x14: [mem 0xfebc5000-0xfebc5fff] [ 1.139040] pci 0000:00:0a.0: reg 0x20: [mem 0xe0000018000-0xe000001bfff 64bit pref] [ 1.164440] ACPI: PCI: Interrupt link LNKA configured for IRQ 10 [ 1.166297] ACPI: PCI: Interrupt link LNKB configured for IRQ 10 [ 1.167243] ACPI: PCI: Interrupt link LNKC configured for IRQ 11 [ 1.169263] ACPI: PCI: Interrupt link LNKD configured for IRQ 11 [ 1.170159] ACPI: PCI: Interrupt link LNKS configured for IRQ 9 [ 1.173319] iommu: Default domain type: Passthrough [ 1.174000] SCSI subsystem initialized [ 1.174000] ACPI: bus type USB registered [ 1.174000] usbcore: registered new interface driver usbfs [ 1.174000] usbcore: registered new interface driver hub [ 1.174000] usbcore: registered new device driver usb [ 1.174000] pps_core: LinuxPPS API ver. 1 registered [ 1.176013] pps_core: Software ver. 5.3.6 - Copyright 2005-2007 Rodolfo Giometti [ 1.178081] PTP clock support registered [ 1.180081] EDAC MC: Ver: 3.0.0 [ 1.183004] PCI: Using ACPI for IRQ routing [ 1.187157] NetLabel: Initializing [ 1.189016] NetLabel: domain hash size = 128 [ 1.190000] NetLabel: protocols = UNLABELED CIPSOv4 CALIPSO [ 1.191205] NetLabel: unlabeled traffic allowed by default [ 1.194554] vgaarb: loaded [ 1.196900] hpet0: at MMIO 0xfed00000, IRQs 2, 8, 0 [ 1.199017] hpet0: 3 comparators, 64-bit 100.000000 MHz counter [ 1.206332] clocksource: Switched to clocksource kvm-clock [ 1.335801] VFS: Disk quotas dquot_6.6.0 [ 1.338528] VFS: Dquot-cache hash table entries: 512 (order 0, 4096 bytes) [ 1.341630] *** VALIDATE ramfs *** [ 1.343248] *** VALIDATE hugetlbfs *** [ 1.345235] pnp: PnP ACPI init [ 1.348176] pnp: PnP ACPI: found 6 devices [ 1.365287] clocksource: acpi_pm: mask: 0xffffff max_cycles: 0xffffff, max_idle_ns: 2085701024 ns [ 1.369084] pci_bus 0000:00: resource 4 [io 0x0000-0x0cf7 window] [ 1.371623] pci_bus 0000:00: resource 5 [io 0x0d00-0xffff window] [ 1.374136] pci_bus 0000:00: resource 6 [mem 0x000a0000-0x000bffff window] [ 1.377057] pci_bus 0000:00: resource 7 [mem 0xc0000000-0xfebfffff window] [ 1.379862] pci_bus 0000:00: resource 8 [mem 0xe0000000000-0xe007fffffff window] [ 1.383173] NET: Registered protocol family 2 [ 1.385882] IP idents hash table entries: 131072 (order: 8, 1048576 bytes, vmalloc) [ 1.391668] tcp_listen_portaddr_hash hash table entries: 4096 (order: 5, 163840 bytes, vmalloc) [ 1.396246] TCP established hash table entries: 65536 (order: 7, 524288 bytes, vmalloc) [ 1.405413] TCP bind hash table entries: 65536 (order: 9, 2097152 bytes, vmalloc) [ 1.410606] TCP: Hash tables configured (established 65536 bind 65536) [ 1.414786] MPTCP token hash table entries: 8192 (order: 6, 393216 bytes, vmalloc) [ 1.419441] UDP hash table entries: 4096 (order: 6, 393216 bytes, vmalloc) [ 1.422924] UDP-Lite hash table entries: 4096 (order: 6, 393216 bytes, vmalloc) [ 1.426719] NET: Registered protocol family 1 [ 1.430413] RPC: Registered named UNIX socket transport module. [ 1.433431] RPC: Registered udp transport module. [ 1.435645] RPC: Registered tcp transport module. [ 1.438060] RPC: Registered tcp NFSv4.1 backchannel transport module. [ 1.441166] NET: Registered protocol family 44 [ 1.443441] pci 0000:00:00.0: Limiting direct PCI/PCI transfers [ 1.445990] pci 0000:00:01.0: PIIX3: Enabling Passive Release [ 1.448676] pci 0000:00:01.0: Activating ISA DMA hang workarounds [ 1.450971] PCI: CLS 0 bytes, default 64 [ 1.452504] Unpacking initramfs... [ 3.183438] debug: unmapping init [mem 0xffff9c927cc54000-0xffff9c927ffbffff] [ 3.190895] PCI-DMA: Using software bounce buffering for IO (SWIOTLB) [ 3.193586] software IO TLB: mapped [mem 0x00000000a8000000-0x00000000ac000000] (64MB) [ 3.198851] clocksource: tsc: mask: 0xffffffffffffffff max_cycles: 0x229835b7123, max_idle_ns: 440795242976 ns [ 3.783727] Initialise system trusted keyrings [ 3.785617] Key type blacklist registered [ 3.787784] workingset: timestamp_bits=36 max_order=20 bucket_order=0 [ 3.797663] zbud: loaded [ 3.800910] *** VALIDATE nfs *** [ 3.802430] *** VALIDATE nfs4 *** [ 3.804483] pstore: using deflate compression [ 3.866132] Platform Keyring initialized [ 3.984735] NET: Registered protocol family 38 [ 3.987065] Key type asymmetric registered [ 3.988358] Asymmetric key parser 'x509' registered [ 3.990607] Block layer SCSI generic (bsg) driver version 0.4 loaded (major 247) [ 3.994193] io scheduler mq-deadline registered [ 3.996155] io scheduler kyber registered [ 3.998396] io scheduler bfq registered [ 4.000489] atomic64_test: passed for x86-64 platform with CX8 and with SSE [ 4.003860] shpchp: Standard Hot Plug PCI Controller Driver version: 0.4 [ 4.007379] input: Power Button as /devices/LNXSYSTM:00/LNXPWRBN:00/input/input0 [ 4.010652] ACPI: Power Button [PWRF] [ 4.016621] ACPI: \_SB_.LNKB: Enabled at IRQ 10 [ 4.024547] ACPI: \_SB_.LNKA: Enabled at IRQ 11 [ 4.088560] ACPI: \_SB_.LNKC: Enabled at IRQ 11 [ 4.097673] ACPI: \_SB_.LNKD: Enabled at IRQ 10 [ 4.270431] Serial: 8250/16550 driver, 4 ports, IRQ sharing enabled [ 4.304329] 00:03: ttyS1 at I/O 0x2f8 (irq = 3, base_baud = 115200) is a 16550A [ 4.334320] 00:04: ttyS0 at I/O 0x3f8 (irq = 4, base_baud = 115200) is a 16550A [ 4.416987] Non-volatile memory driver v1.3 [ 4.418536] Linux agpgart interface v0.103 [ 4.518154] virtio_blk virtio1: [vda] 146144 512-byte logical blocks (74.8 MB/71.4 MiB) [ 4.522853] vda: detected capacity change from 0 to 74825728 [ 4.572880] virtio_blk virtio2: [vdb] 2097152 512-byte logical blocks (1.07 GB/1.00 GiB) [ 4.575702] vdb: detected capacity change from 0 to 1073741824 [ 4.602208] virtio_blk virtio3: [vdc] 5120000 512-byte logical blocks (2.62 GB/2.44 GiB) [ 4.605599] vdc: detected capacity change from 0 to 2621440000 [ 4.619807] virtio_blk virtio4: [vdd] 5120000 512-byte logical blocks (2.62 GB/2.44 GiB) [ 4.622049] vdd: detected capacity change from 0 to 2621440000 [ 4.639392] virtio_blk virtio5: [vde] 8388608 512-byte logical blocks (4.29 GB/4.00 GiB) [ 4.642619] vde: detected capacity change from 0 to 4294967296 [ 4.665153] virtio_blk virtio6: [vdf] 8388608 512-byte logical blocks (4.29 GB/4.00 GiB) [ 4.669649] vdf: detected capacity change from 0 to 4294967296 [ 4.680340] libphy: Fixed MDIO Bus: probed [ 4.754288] usbcore: registered new interface driver usbserial_generic [ 4.763385] usbserial: USB Serial support registered for generic [ 4.772656] i8042: PNP: PS/2 Controller [PNP0303:KBD,PNP0f13:MOU] at 0x60,0x64 irq 1,12 [ 4.787985] serio: i8042 KBD port at 0x60,0x64 irq 1 [ 4.791895] serio: i8042 AUX port at 0x60,0x64 irq 12 [ 4.801765] mousedev: PS/2 mouse device common for all mice [ 4.804930] input: AT Translated Set 2 keyboard as /devices/platform/i8042/serio0/input/input1 [ 4.806390] rtc_cmos 00:05: RTC can wake from S4 [ 4.813706] rtc_cmos 00:05: registered as rtc0 [ 4.815394] rtc_cmos 00:05: alarms up to one day, y3k, 242 bytes nvram, hpet irqs [ 4.819229] intel_pstate: CPU model not supported [ 4.822608] hid: raw HID events driver (C) Jiri Kosina [ 4.824825] usbcore: registered new interface driver usbhid [ 4.827341] usbhid: USB HID core driver [ 4.829142] drop_monitor: Initializing network drop monitor service [ 4.829712] input: VirtualPS/2 VMware VMMouse as /devices/platform/i8042/serio1/input/input4 [ 4.834114] Initializing XFRM netlink socket [ 4.838404] NET: Registered protocol family 10 [ 4.841963] Segment Routing with IPv6 [ 4.843434] NET: Registered protocol family 17 [ 4.845588] mpls_gso: MPLS GSO support [ 4.848640] input: VirtualPS/2 VMware VMMouse as /devices/platform/i8042/serio1/input/input3 [ 4.858649] RAS: Correctable Errors collector initialized. [ 4.860821] AVX version of gcm_enc/dec engaged. [ 4.862635] AES CTR mode by8 optimization enabled [ 5.159781] sched_clock: Marking stable (5159700391, 0)->(6379752379, -1220051988) [ 5.170655] registered taskstats version 1 [ 5.174488] Loading compiled-in X.509 certificates [ 5.176888] zswap: loaded using pool lzo/zbud [ 5.205250] Key type big_key registered [ 5.236161] Key type encrypted registered [ 5.239275] ima: No TPM chip found, activating TPM-bypass! [ 5.241903] ima: Allocated hash algorithm: sha1 [ 5.245653] ima: No architecture policies found [ 5.248705] evm: Initialising EVM extended attributes: [ 5.252513] evm: security.selinux [ 5.255111] evm: security.ima [ 5.257377] evm: security.capability [ 5.260369] evm: HMAC attrs: 0x1 [ 5.267458] rtc_cmos 00:05: setting system clock to 2026-08-22 20:56:33 UTC (1787432193) [ 5.279484] debug: unmapping init [mem 0xffffffffb9003000-0xffffffffb91fffff] [ 5.285676] debug: unmapping init [mem 0xffffffffb7d82000-0xffffffffb8058fff] [ 5.295649] Write protecting the kernel read-only data: 28672k [ 5.301634] debug: unmapping init [mem 0xffffffffb6403000-0xffffffffb65fffff] [ 5.305817] debug: unmapping init [mem 0xffffffffb6d14000-0xffffffffb6dfffff] [ 5.374694] systemd[1]: systemd 239 (239-82.el8_10.5) running in system mode. (+PAM +AUDIT +SELINUX +IMA -APPARMOR +SMACK +SYSVINIT +UTMP +LIBCRYPTSETUP +GCRYPT +GNUTLS +ACL +XZ +LZ4 +SECCOMP +BLKID +ELFUTILS +KMOD +IDN2 -IDN +PCRE2 default-hierarchy=legacy) [ 5.392503] systemd[1]: Detected virtualization kvm. [ 5.397350] systemd[1]: Detected architecture x86-64. [ 5.401052] systemd[1]: Running in initial RAM disk. Welcome to Rocky Linux 8.10 (Green Obsidian) dracut-049-233.git20240115.el8 (Initramfs)! [ 5.440511] systemd[1]: No hostname configured. [ 5.447428] systemd[1]: Set hostname to . [ 5.452947] random: systemd: uninitialized urandom read (16 bytes read) [ 5.457488] systemd[1]: Initializing machine ID from random generator. [ 5.667977] random: ln: uninitialized urandom read (6 bytes read) [ 6.260024] random: systemd: uninitialized urandom read (16 bytes read) [ 6.265545] systemd[1]: Reached target Timers. [ OK ] Reached target Timers. [ 6.399186] systemd[1]: Reached target Slices. [ OK ] Reached target Slices. [ 6.409759] systemd[1]: Reached target Swap. [ OK ] Reached target Swap. [ OK ] Listening on udev Control Socket. [ OK ] Reached target Initrd Root Device. [ OK ] Listening on Journal Socket. Starting Setup Virtual Console... Starting Apply Kernel Variables... [ OK ] Started Memstrack Anylazing Service. [ OK ] Reached target Local File Systems. [ OK ] Listening on udev Kernel Socket. Starting Create list of required st…ce nodes for the current kernel... [ OK ] Started Dispatch Password Requests to Console Directory Watch. [ OK ] Reached target Paths. [ OK ] Reached target Local Encrypted Volumes. Starting Create Volatile Files and Directories... [ OK ] Listening on Journal Socket (/dev/log). [ OK ] Reached target Sockets. Starting Journal Service... [ OK ] Started Setup Virtual Console. [ OK ] Started Apply Kernel Variables. [ OK ] Started Create list of required sta…vice nodes for the current kernel. [ OK ] Started Create Volatile Files and Directories. Starting Create Static Device Nodes in /dev... Starting dracut cmdline hook... [ OK ] Started Create Static Device Nodes in /dev. [ OK ] Started Journal Service. [ OK ] Started dracut cmdline hook. Starting dracut pre-udev hook... [ 7.726619] device-mapper: uevent: version 1.0.3 [ 7.729303] device-mapper: ioctl: 4.46.0-ioctl (2022-02-22) initialised: dm-devel@redhat.com [ OK ] Started dracut pre-udev hook. Starting udev Kernel Device Manager... [ OK ] Started udev Kernel Device Manager. Starting dracut pre-trigger hook... [ OK ] Started dracut pre-trigger hook. Starting udev Coldplug all Devices... [ OK ] Started udev Coldplug all Devices. Mounting Kernel Configuration File System... Starting dracut initqueue hook... [ 11.344918] virtio_net virtio0 ens2: renamed from eth0 [ OK ] Mounted Kernel Configuration File System. [ 11.656956] random: fast init done [ 11.995285] scsi host0: ata_piix [ 12.061304] scsi host1: ata_piix [ 12.064393] ata1: PATA max MWDMA2 cmd 0x1f0 ctl 0x3f6 bmdma 0xc320 irq 14 [ 12.067595] ata2: PATA max MWDMA2 cmd 0x170 ctl 0x376 bmdma 0xc328 irq 15 [ OK ] Reached target System Initialization. [ OK ] Reached target Basic System. [ OK ] Started Hardware RNG Entropy Gatherer Daemon. [ 17.790749] random: crng init done [ 17.796928] random: 7 urandom warning(s) missed due to ratelimiting [ 18.107259] dracut-initqueue[567]: RTNETLINK answers: File exists Starting nbd nbd0... [ OK ] Started nbd nbd0. [ OK ] Started dracut initqueue hook. Mounting /sysroot... [ OK ] Reached target Remote File Systems (Pre). [ OK ] Reached target Remote File Systems. [ 21.082367] EXT4-fs (nbd0): mounted filesystem with ordered data mode. Opts: (null) [ OK ] Mounted /sysroot. [ OK ] Reached target Initrd Root File System. Starting Reload Configuration from the Real Root... [ OK ] Started Reload Configuration from the Real Root. [ OK ] Reached target Initrd File Systems. [ OK ] Reached target Initrd Default Target. Starting dracut pre-pivot and cleanup hook... [ OK ] Started dracut pre-pivot and cleanup hook. Starting Cleaning Up and Shutting Down Daemons... [ OK ] Stopped target Timers. Stopping Hardware RNG Entropy Gatherer Daemon... [ OK ] Stopped dracut pre-pivot and cleanup hook. [ OK ] Stopped target Remote File Systems. [ OK ] Stopped target Remote File Systems (Pre). [ OK ] Stopped target Initrd Default Target. [ OK ] Stopped dracut initqueue hook. [ OK ] Stopped target Initrd Root Device. [ OK ] Stopped Hardware RNG Entropy Gatherer Daemon. [ OK ] Stopped target Basic System. [ OK ] Stopped target Slices. [ OK ] Stopped target Sockets. [ OK ] Stopped target Paths. [ OK ] Stopped target System Initialization. [ OK ] Stopped target Swap. [ OK ] Stopped target Local Encrypted Volumes. [ OK ] Stopped Dispatch Password Requests to Console Directory Watch. [ OK ] Stopped Apply Kernel Variables. [ OK ] Stopped Create Volatile Files and Directories. [ OK ] Stopped target Local File Systems. [ OK ] Stopped udev Coldplug all Devices. [ OK ] Stopped dracut pre-trigger hook. Stopping udev Kernel Device Manager... [ OK ] Stopped udev Kernel Device Manager. [ OK ] Started Cleaning Up and Shutting Down Daemons. [ OK ] Stopped dracut pre-udev hook. [ OK ] Stopped dracut cmdline hook. [ OK ] Stopped Create Static Device Nodes in /dev. [ OK ] Stopped Create list of required sta…vice nodes for the current kernel. [ OK ] Closed udev Control Socket. [ OK ] Closed udev Kernel Socket. Starting Cleanup udevd DB... [ OK ] Started Cleanup udevd DB. [ OK ] Reached target Switch Root. Starting Switch Root... [ 25.456069] printk: systemd: 26 output lines suppressed due to ratelimiting [ 26.398890] SELinux: Disabled at runtime. [ 26.726343] systemd[1]: systemd 239 (239-82.el8_10.5) running in system mode. (+PAM +AUDIT +SELINUX +IMA -APPARMOR +SMACK +SYSVINIT +UTMP +LIBCRYPTSETUP +GCRYPT +GNUTLS +ACL +XZ +LZ4 +SECCOMP +BLKID +ELFUTILS +KMOD +IDN2 -IDN +PCRE2 default-hierarchy=legacy) [ 26.751326] systemd[1]: Detected virtualization kvm. [ 26.760011] systemd[1]: Detected architecture x86-64. Welcome to Rocky Linux 8.10 (Green Obsidian)! [ 29.118409] systemd[1]: initrd-switch-root.service: Succeeded. [ 29.141641] systemd[1]: Stopped Switch Root. [ OK ] Stopped Switch Root. [ 29.159053] systemd[1]: systemd-journald.service: Service has no hold-off time (RestartSec=0), scheduling restart. [ 29.166473] systemd[1]: systemd-journald.service: Scheduled restart job, restart counter is at 1. [ 29.172524] systemd[1]: Stopped Journal Service. [ OK ] Stopped Journal Service. [ 29.196170] systemd[1]: Starting Journal Service... Starting Journal Service... [ 29.224874] systemd[1]: proc-sys-fs-binfmt_misc.automount: Refusing to start, unit to trigger not loaded. [FAILED] Failed to set up automount Arbitrar…rmats File System Automount Point. See 'systemctl status proc-sys-fs-binfmt_misc.automount' for details. [ OK ] Created slice system-serial\x2dgetty.slice. [ OK ] Reached target rpc_pipefs.target. [ OK ] Started Dispatch Password Requests to Console Directory Watch. [ OK ] Listening on initctl Compatibility Named Pipe. Mounting Kernel Debug File System... [ OK ] Stopped target Switch Root. [ OK ] Listening on RPCbind Server Activation Socket. [ OK ] Created slice system-sshd\x2dkeygen.slice. [ OK ] Stopped target Initrd Root File System. Starting Create list of required st…ce nodes for the current kernel... [ OK ] Started Forward Password Requests to Wall Directory Watch. [ OK ] Reached target Paths. Starting Apply Kernel Variables... [ OK ] Created slice User and Session Slice. [ OK ] Reached target Slices. Starting Remount Root and Kernel File Systems... [ OK ] Reached target RPC Port Mapper. [ OK ] Listening on udev Kernel Socket. [ OK ] Reached target Local Encrypted Volumes. Mounting POSIX Message Queue File System... Activating swap /dev/disk/by-label/SWAP... Mounting Huge Pages File System... [ OK ] Stopped target Initrd File Systems. [ 29.813640] Adding 1048572k swap on /dev/vdb. Priority:-2 extents:1 across:1048572k FS [ OK ] Listening on Process Core Dump Socket. [ OK ] Created slice system-getty.slice. [ OK ] Listening on udev Control Socket. Starting udev Coldplug all Devices... [ OK ] Started Journal Service. [ OK ] Mounted Kernel Debug File System. [ OK ] Started Create list of required sta…vice nodes for the current kernel. [ OK ] Started Apply Kernel Variables. [FAILED] Failed to start Remount Root and Kernel File Systems. See 'systemctl status systemd-remount-fs.service' for details. [ OK ] Mounted POSIX Message Queue File System. [ OK ] Activated swap /dev/disk/by-label/SWAP. [ OK ] Mounted Huge Pages File System. [ OK ] Reached target Swap. Starting Configure read-only root support... Starting Create Static Device Nodes in /dev... Starting Flush Journal to Persistent Storage... [ OK ] Started Flush Journal to Persistent Storage. [ OK ] Started udev Coldplug all Devices. [ OK ] Started Create Static Device Nodes in /dev. Starting udev Kernel Device Manager... [ OK ] Reached target Local File Systems (Pre). Mounting /mnt... Mounting /home/green/git/lustre-release... [ OK ] Mounted /mnt. [ 31.242075] squashfs: version 4.0 (2009/01/31) Phillip Lougher [ OK ] Mounted /home/green/git/lustre-release. [ OK ] Started udev Kernel Device Manager. [ 32.532462] piix4_smbus 0000:00:01.3: SMBus Host Controller at 0x700, revision 0 [ 32.612345] input: PC Speaker as /devices/platform/pcspkr/input/input5 [ 33.415750] RAPL PMU: API unit is 2^-32 Joules, 0 fixed counters, 10737418240 ms ovfl timer [ 33.569582] EDAC sbridge: Ver: 1.1.2 [* ] A start job is running for Configur…-only root support (8s / no limit) [** ] A start job is running for Configur…-only root support (8s / no limit) [*** ] A start job is running for Configur…-only root support (9s / no limit) [ *** ] A start job is running for Configur…only root support (10s / no limit) [ *** ] A start job is running for Configur…only root support (10s / no limit)[ 39.527135] Key type dns_resolver registered [ ***] A start job is running for Configur…only root support (11s / no limit) [ **] A start job is running for Configur…only root support (11s / no limit)[ 40.694204] NFS: Registering the id_resolver key type [ 40.696535] Key type id_resolver registered [ 40.698598] Key type id_legacy registered [ *] A start job is running for Configur…only root support (12s / no limit) [ **] A start job is running for Configur…only root support (12s / no limit) [ OK ] Started Configure read-only root support. Starting Load/Save Random Seed... [ OK ] Reached target Local File Systems. Starting Mark the need to relabel after reboot... Starting Rebuild Dynamic Linker Cache... Starting Create Volatile Files and Directories... [ OK ] Started Load/Save Random Seed. [ OK ] Started Mark the need to relabel after reboot. [ OK ] Started Create Volatile Files and Directories. Starting RPC Bind... Starting Update UTMP about System Boot/Shutdown... [ OK ] Started Update UTMP about System Boot/Shutdown. [ OK ] Started RPC Bind. [ OK ] Started Rebuild Dynamic Linker Cache. Starting Update is Completed... [ OK ] Started Update is Completed. [ OK ] Reached target System Initialization. [ OK ] Started daily update of the root trust anchor for DNSSEC. [ OK ] Listening on D-Bus System Message Bus Socket. [ OK ] Reached target Sockets. [ OK ] Reached target Basic System. [ OK ] Reached target sshd-keygen.target. [ OK ] Started irqbalance daemon. [ OK ] Started D-Bus System Message Bus. Starting Restore /run/initramfs on shutdown... [ OK ] Started dnf makecache --timer. [ OK ] Started Daily Cleanup of Temporary Directories. [ OK ] Reached target Timers. Starting Network Manager... Starting Login Service... [ OK ] Started Hardware RNG Entropy Gatherer Daemon. [ OK ] Started Restore /run/initramfs on shutdown. [ OK ] Started Network Manager. [ OK ] Reached target Network. Starting Dynamic System Tuning Daemon... Starting OpenSSH server daemon... Starting GSSAPI Proxy Daemon... Starting Network Manager Wait Online... Starting Hostname Service... [ OK ] Started OpenSSH server daemon. [ OK ] Started Login Service. [ OK ] Started GSSAPI Proxy Daemon. [ OK ] Reached target NFS client services. [ OK ] Reached target Remote File Systems (Pre). [ OK ] Reached target Remote File Systems. Starting Permit User Sessions... [ OK ] Started Permit User Sessions. [ OK ] Started Serial Getty on ttyS0. [ OK ] Started Serial Getty on ttyS1. [ OK ] Started Command Scheduler. [ OK ] Started Getty on tty1. [ OK ] Reached target Login Prompts. [ OK ] Started Hostname Service. Starting Network Manager Script Dispatcher Service... [ OK ] Started Network Manager Script Dispatcher Service. [ OK ] Started Network Manager Wait Online. [ OK ] Reached target Network is Online. Starting Notify NFS peers of a restart... Starting Crash recovery kernel arming... Starting System Logging Service... [ OK ] Started Notify NFS peers of a restart. Rocky Linux 8.10 (Green Obsidian) Kernel 4.18.0rh8.10-debug on an x86_64 oleg211-server login: [ 109.885993] libcfs: loading out-of-tree module taints kernel. [ 109.948820] Key type ._llcrypt registered [ 109.951625] Key type .llcrypt registered [ 110.065161] Lustre: DEBUG MARKER: oleg211-server.virtnet: executing set_hostid [ 129.539447] Lustre: DEBUG MARKER: oleg211-server.virtnet: executing load_modules_local [ 131.023302] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 131.039819] alg: No test for adler32 (adler32-zlib) [ 132.755555] Lustre: Lustre: Build Version: 2.17.57_63_gaeb0ce0 [ 133.742815] LNet: Added LNI 192.168.202.111@tcp [8/256/0/180] [ 135.519490] Key type lgssc registered [ 137.181394] Lustre: Echo OBD driver; http://www.lustre.org/ [ 151.408707] ZFS: Loaded module v2.3.2-1, ZFS pool version 5000, ZFS filesystem version 5 [ 185.445272] hrtimer: interrupt took 2957034 ns [ 204.127418] Lustre: DEBUG MARKER: oleg211-server.virtnet: executing load_modules_local [ 220.451948] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 220.527902] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 221.888618] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 221.915779] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 222.063075] Lustre: lustre-MDT0000: new disk, initializing [ 222.184639] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 222.232577] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 227.229071] Lustre: DEBUG MARKER: oleg211-server.virtnet: executing set_default_debug vfstrace rpctrace dlmtrace neterror ha config ioctl super lfsck all [ 243.152331] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 243.260726] Lustre: 6517:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 243.290969] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [ 243.294787] Lustre: Skipped 1 previous similar message [ 243.359964] Lustre: lustre-MDT0001: new disk, initializing [ 243.425479] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 243.474385] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [ 243.534090] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 248.617505] Lustre: DEBUG MARKER: oleg211-server.virtnet: executing set_default_debug vfstrace rpctrace dlmtrace neterror ha config ioctl super lfsck all [ 253.859243] Lustre: Modifying parameter general.debug_raw_pointers=Y in log params [ 264.391514] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 264.880983] Lustre: lustre-OST0000: new disk, initializing [ 264.884550] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 264.896451] Lustre: 8454:0:(osd_compat.c:1353:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 264.957612] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 265.870184] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [ 265.882499] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 265.968619] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 271.558547] Lustre: DEBUG MARKER: oleg211-server.virtnet: executing set_default_debug vfstrace rpctrace dlmtrace neterror ha config ioctl super lfsck all [ 289.392225] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 289.574377] Lustre: lustre-OST0001: new disk, initializing [ 289.578475] Lustre: srv-lustre-OST0001: No data found on store. Initialize space. [ 289.588348] Lustre: 9539:0:(osd_compat.c:1353:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 289.662460] Lustre: lustre-OST0001: Imperative Recovery not enabled, recovery window 60-180 [ 296.349659] Lustre: DEBUG MARKER: oleg211-server.virtnet: executing set_default_debug vfstrace rpctrace dlmtrace neterror ha config ioctl super lfsck all [ 297.514103] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x00000002c0000400-0x0000000300000400]:1:ost [ 297.523726] Lustre: cli-lustre-OST0001-super: Allocated super-sequence [0x00000002c0000400-0x0000000300000400]:1:ost] [ 297.570043] Lustre: lustre-OST0001-osc-MDT0000: update sequence from 0x100010000 to 0x2c0000401 [ 308.507713] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 314.557766] Lustre: Setting parameter general.lod.*.mdt_hash=crush in log params [ 321.191255] Lustre: DEBUG MARKER: oleg211-server.virtnet: executing check_logdir /tmp/testlogs/ [ 326.925909] Lustre: DEBUG MARKER: oleg211-server.virtnet: executing yml_node [ 331.790274] Lustre: DEBUG MARKER: Client: 2.17.57.63 [ 335.339796] Lustre: DEBUG MARKER: MDS: 2.17.57.63 [ 338.576074] Lustre: DEBUG MARKER: OSS: 2.17.57.63 [ 340.542753] Lustre: DEBUG MARKER: -----============= acceptance-small: conf-sanity ============----- Sat Aug 22 17:02:06 EDT 2026 [ 361.153510] Lustre: DEBUG MARKER: excepting tests: 102 106 115 32newtarball [ 362.780652] Lustre: DEBUG MARKER: skipping tests SLOW=no: 45 69 106 111 114 [ 379.361208] Lustre: lustre-MDT0000-lwp-OST0000: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 379.373353] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 379.395262] Lustre: Skipped 3 previous similar messages [ 379.411941] Lustre: Skipped 3 previous similar messages [ 384.482486] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 384.492860] Lustre: Skipped 3 previous similar messages [ 389.604833] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 389.621336] Lustre: Skipped 3 previous similar messages [ 393.183211] Lustre: lustre-MDT0000 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 393.408341] Lustre: server umount lustre-MDT0000 complete [ 397.250306] LustreError: 7472:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1787432585 with bad export cookie 2640131335954330246 [ 397.255334] LustreError: MGC192.168.202.111@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 397.262593] LustreError: 7472:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 3 previous similar messages [ 399.840618] LustreError: 6522:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 399.843774] Lustre: lustre-MDT0001-lwp-OST0001: Connection to lustre-MDT0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 399.852080] Lustre: lustre-MDT0001: Not available for connect from 0@lo (stopping) [ 399.862929] LustreError: 6522:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 6 previous similar messages [ 399.916053] Lustre: Skipped 1 previous similar message [ 402.848207] LustreError: 6523:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 402.876611] LustreError: 6523:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 1 previous similar message [ 403.918983] Lustre: server umount lustre-MDT0001 complete [ 408.279965] Lustre: server umount lustre-OST0000 complete [ 412.241887] Lustre: server umount lustre-OST0001 complete [ 419.194764] Lustre: DEBUG MARKER: oleg211-server.virtnet: executing set_hostid [ 428.339552] Lustre: DEBUG MARKER: oleg211-server.virtnet: executing load_modules_local [ 466.612209] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 466.883919] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 466.939482] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 467.019407] Lustre: lustre-MDT0000: new disk, initializing [ 467.091086] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 467.101624] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 472.264810] Lustre: DEBUG MARKER: oleg211-server.virtnet: executing set_default_debug -1 all [ 482.561936] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 482.640706] Lustre: 15097:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 482.693244] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [ 482.696653] Lustre: Skipped 1 previous similar message [ 482.758231] Lustre: lustre-MDT0001: new disk, initializing [ 482.869612] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 482.912873] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [ 482.928089] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 487.368694] Lustre: DEBUG MARKER: oleg211-server.virtnet: executing set_default_debug -1 all [ 498.936456] Lustre: DEBUG MARKER: oleg211-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 506.292354] Lustre: DEBUG MARKER: oleg211-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 513.231780] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 513.547335] Lustre: lustre-OST0000: new disk, initializing [ 513.552117] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 513.558881] Lustre: 17066:0:(osd_compat.c:1353:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 513.692619] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 515.329657] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [ 515.341236] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 515.540743] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 519.938010] Lustre: DEBUG MARKER: oleg211-server.virtnet: executing set_default_debug -1 all [ 530.862043] Lustre: DEBUG MARKER: oleg211-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 535.574527] Lustre: DEBUG MARKER: oleg211-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 [ 535.820993] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [ 541.018626] Lustre: DEBUG MARKER: oleg211-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid 50 [ 541.288850] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [ 546.283170] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 546.298811] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 546.302316] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 546.325490] Lustre: Skipped 1 previous similar message [ 546.349681] Lustre: Skipped 4 previous similar messages [ 549.436217] Lustre: server umount lustre-OST0000 complete [ 556.524079] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 556.524078] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 556.525443] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 556.559439] Lustre: Skipped 1 previous similar message [ 561.642645] LustreError: 15102:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 561.668031] LustreError: 15102:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 1 previous similar message [ 561.793457] Lustre: server umount lustre-MDT0000 complete [ 566.328249] LustreError: 17065:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1787432754 with bad export cookie 2640131335954331779 [ 566.340767] LustreError: MGC192.168.202.111@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 566.343719] LustreError: 17065:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 3 previous similar messages [ 566.988372] Lustre: server umount lustre-MDT0001 complete [ 573.166894] Lustre: DEBUG MARKER: == conf-sanity test 76a: set permanent params with lctl across mounts ========================================================== 17:05:59 (1787432759) [ 580.634458] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 581.268389] LustreError: 19330:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 581.389155] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 586.727922] LustreError: 19331:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 586.821625] Lustre: DEBUG MARKER: oleg211-server.virtnet: executing set_default_debug -1 all [ 595.949625] LustreError: 19331:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 595.993884] LustreError: 19331:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 1 previous similar message [ 596.639192] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 602.856717] Lustre: DEBUG MARKER: oleg211-server.virtnet: executing set_default_debug -1 all [ 611.153995] Lustre: DEBUG MARKER: oleg211-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 617.543893] Lustre: DEBUG MARKER: oleg211-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 624.427815] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 630.368843] Lustre: DEBUG MARKER: oleg211-server.virtnet: executing set_default_debug -1 all [ 637.814919] Lustre: DEBUG MARKER: oleg211-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 644.445359] Lustre: Modifying parameter general.osc.*.max_dirty_mb=452 in log params [ 652.770897] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 652.778082] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 652.791141] Lustre: Skipped 1 previous similar message [ 656.831294] Lustre: server umount lustre-MDT0000 complete [ 657.895590] LustreError: 19330:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 657.911791] LustreError: 19330:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 2 previous similar messages [ 660.636294] LustreError: 19312:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1787432848 with bad export cookie 2640131335954332661 [ 660.661580] LustreError: MGC192.168.202.111@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 661.007277] Lustre: server umount lustre-MDT0001 complete [ 675.378138] Lustre: server umount lustre-OST0000 complete [ 701.572541] Lustre: DEBUG MARKER: oleg211-server.virtnet: executing load_modules_local [ 712.619969] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 712.915353] LustreError: 23714:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 712.984082] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 712.993962] Lustre: Skipped 2 previous similar messages [ 717.759916] Lustre: DEBUG MARKER: oleg211-server.virtnet: executing set_default_debug -1 all [ 728.162793] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 735.391923] Lustre: DEBUG MARKER: oleg211-server.virtnet: executing set_default_debug -1 all [ 739.064948] Lustre: Setting parameter general.debug_raw_pointers=Y in log params [ 747.290978] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 754.813967] Lustre: DEBUG MARKER: oleg211-server.virtnet: executing set_default_debug -1 all [ 765.013227] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 765.207589] Lustre: lustre-OST0001: new disk, initializing [ 765.210885] Lustre: srv-lustre-OST0001: No data found on store. Initialize space. [ 765.215554] Lustre: 26031:0:(osd_compat.c:1353:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 766.441733] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x00000002c0000400-0x0000000300000400]:1:ost [ 766.468632] Lustre: cli-lustre-OST0001-super: Allocated super-sequence [0x00000002c0000400-0x0000000300000400]:1:ost] [ 766.675534] Lustre: lustre-OST0001-osc-MDT0000: update sequence from 0x100010000 to 0x2c0000401 [ 772.044304] Lustre: DEBUG MARKER: oleg211-server.virtnet: executing set_default_debug -1 all [ 782.441451] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 786.504482] Lustre: Setting parameter general.lod.*.mdt_hash=crush in log params [ 800.224363] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 800.231528] LustreError: Skipped 1 previous similar message [ 800.236350] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 800.254425] Lustre: Skipped 1 previous similar message [ 800.266710] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 800.271849] Lustre: Skipped 2 previous similar messages [ 805.485572] Lustre: server umount lustre-MDT0000 complete [ 807.394340] LustreError: 23709:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 807.440234] LustreError: 23709:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 5 previous similar messages [ 809.761978] LustreError: 23695:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1787432997 with bad export cookie 2640131335954333508 [ 809.763246] LustreError: MGC192.168.202.111@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 809.799324] LustreError: 23695:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 7 previous similar messages [ 810.436540] Lustre: server umount lustre-MDT0001 complete [ 824.714789] Lustre: server umount lustre-OST0000 complete [ 827.872201] Lustre: 3628:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1787433000/real 1787433000] req@ffff9c9300e97b80 x1874258633673472/t0(0) o400->lustre-MDT0001-lwp-OST0001@0@lo:12/10 lens 224/224 e 0 to 1 dl 1787433016 ref 1 fl Rpc:XNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 [ 828.911562] Lustre: server umount lustre-OST0001 complete [ 847.926152] Lustre: DEBUG MARKER: oleg211-server.virtnet: executing load_modules_local [ 858.635458] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 859.032145] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 859.038203] Lustre: Skipped 3 previous similar messages [ 864.540136] Lustre: DEBUG MARKER: oleg211-server.virtnet: executing set_default_debug -1 all [ 874.327744] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 880.725954] Lustre: DEBUG MARKER: oleg211-server.virtnet: executing set_default_debug -1 all [ 883.826858] Lustre: 30198:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify general/debug_raw_pointers=Y (mode = 0) failed: rc = -17 [ 893.483281] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 899.661828] Lustre: DEBUG MARKER: oleg211-server.virtnet: executing set_default_debug -1 all [ 908.987978] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 917.017877] Lustre: DEBUG MARKER: oleg211-server.virtnet: executing set_default_debug -1 all [ 925.227781] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 928.691249] Lustre: 32077:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify general/lod.*.mdt_hash=crush (mode = 0) failed: rc = -17 [ 934.883098] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 934.887765] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 934.902383] Lustre: Skipped 7 previous similar messages [ 934.917664] Lustre: Skipped 7 previous similar messages [ 940.648274] Lustre: server umount lustre-MDT0000 complete [ 943.013691] LustreError: 29049:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 943.044538] LustreError: 29049:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 11 previous similar messages [ 944.525346] LustreError: 29031:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1787433132 with bad export cookie 2640131335954334803 [ 944.542020] LustreError: MGC192.168.202.111@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 944.542474] LustreError: 29031:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 3 previous similar messages [ 944.903526] Lustre: server umount lustre-MDT0001 complete [ 959.374397] Lustre: server umount lustre-OST0000 complete [ 973.643734] Lustre: server umount lustre-OST0001 complete [ 982.289556] Lustre: DEBUG MARKER: == conf-sanity test 76b: verify params log setup correctly ========================================================== 17:12:48 (1787433168) [ 1012.061861] Lustre: DEBUG MARKER: oleg211-server.virtnet: executing load_modules_local [ 1022.902511] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1027.313695] Lustre: DEBUG MARKER: oleg211-server.virtnet: executing set_default_debug -1 all [ 1037.041294] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1044.641139] Lustre: DEBUG MARKER: oleg211-server.virtnet: executing set_default_debug -1 all [ 1048.508846] Lustre: 36030:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify general/debug_raw_pointers=Y (mode = 0) failed: rc = -17 [ 1055.099045] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 1062.091895] Lustre: DEBUG MARKER: oleg211-server.virtnet: executing set_default_debug -1 all [ 1071.808908] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 1078.400909] Lustre: DEBUG MARKER: oleg211-server.virtnet: executing set_default_debug -1 all [ 1087.008822] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 1090.387690] Lustre: 37909:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify general/lod.*.mdt_hash=crush (mode = 0) failed: rc = -17 [ 1097.698712] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 1097.710816] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 1097.719136] Lustre: Skipped 2 previous similar messages [ 1097.747458] Lustre: Skipped 7 previous similar messages [ 1100.778242] Lustre: server umount lustre-MDT0000 complete [ 1104.648741] LustreError: 34862:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1787433292 with bad export cookie 2640131335954335909 [ 1104.664071] LustreError: MGC192.168.202.111@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 1104.667955] LustreError: 34862:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 1105.106695] Lustre: server umount lustre-MDT0001 complete [ 1119.651073] Lustre: server umount lustre-OST0000 complete [ 1133.634840] Lustre: server umount lustre-OST0001 complete [ 1144.017189] Lustre: DEBUG MARKER: == conf-sanity test 76c: verify changelog_mask is applied with lctl set_param -P ========================================================== 17:15:29 (1787433329) [ 1159.117498] Lustre: DEBUG MARKER: oleg211-server.virtnet: executing load_modules_local [ 1169.881138] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1170.234399] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 1170.242827] Lustre: Skipped 7 previous similar messages [ 1174.306534] Lustre: DEBUG MARKER: oleg211-server.virtnet: executing set_default_debug -1 all [ 1182.834323] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1187.567561] Lustre: DEBUG MARKER: oleg211-server.virtnet: executing set_default_debug -1 all [ 1190.695305] Lustre: 41201:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify general/debug_raw_pointers=Y (mode = 0) failed: rc = -17 [ 1197.415426] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 1203.332269] Lustre: DEBUG MARKER: oleg211-server.virtnet: executing set_default_debug -1 all [ 1208.054297] LustreError: 41555:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 1208.076970] LustreError: 41555:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 13 previous similar messages [ 1211.151765] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 1217.100818] Lustre: DEBUG MARKER: oleg211-server.virtnet: executing set_default_debug -1 all [ 1223.994460] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 1227.402107] Lustre: 43078:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify general/lod.*.mdt_hash=crush (mode = 0) failed: rc = -17 [ 1228.225458] Lustre: Setting parameter general.mdd.*.changelog_mask=-CLOSE in log params [ 1228.229304] Lustre: Skipped 1 previous similar message [ 1230.820204] Lustre: lustre-MDT0000-lwp-OST0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 1230.838869] Lustre: Skipped 3 previous similar messages [ 1236.958869] Lustre: server umount lustre-MDT0000 complete [ 1240.539459] LustreError: 40035:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1787433428 with bad export cookie 2640131335954337008 [ 1240.546854] LustreError: MGC192.168.202.111@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 1246.961084] Lustre: server umount lustre-MDT0001 complete [ 1255.459330] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1265.640629] Lustre: Evicted from MGS (at 0@lo) after server handle changed from 0x0 to 0x24a3a17647ef915e [ 1265.649926] Lustre: MGC192.168.202.111@tcp: Connection restored to 0@lo (at 0@lo) [ 1270.728490] Lustre: DEBUG MARKER: oleg211-server.virtnet: executing set_default_debug -1 all [ 1275.265838] Lustre: lustre-MDT0000-lwp-OST0000: Connection restored to 0@lo (at 0@lo) [ 1279.257987] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1284.083952] Lustre: DEBUG MARKER: oleg211-server.virtnet: executing set_default_debug -1 all [ 1285.102969] Lustre: lustre-MDT0001-lwp-OST0000: Connection restored to 0@lo (at 0@lo) [ 1285.118400] Lustre: Skipped 1 previous similar message [ 1291.905369] Lustre: DEBUG MARKER: oleg211-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 1293.752432] Lustre: DEBUG MARKER: mdc.lustre-MDT0000-mdc-*.mds_server_uuid in FULL state after 0 sec [ 1300.363245] Lustre: DEBUG MARKER: oleg211-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 1302.015749] Lustre: DEBUG MARKER: mdc.lustre-MDT0001-mdc-*.mds_server_uuid in FULL state after 0 sec [ 1310.191108] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 1313.963804] Lustre: server umount lustre-MDT0000 complete [ 1318.453546] LustreError: 43487:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1787433506 with bad export cookie 2640131335954338142 [ 1318.461030] LustreError: 43487:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 5 previous similar messages [ 1318.469544] LustreError: MGC192.168.202.111@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 1318.705919] Lustre: server umount lustre-MDT0001 complete [ 1332.873448] Lustre: server umount lustre-OST0000 complete [ 1346.168860] Lustre: server umount lustre-OST0001 complete [ 1354.878268] Lustre: DEBUG MARKER: == conf-sanity test 76d: verify llite.*.xattr_cache can be set by 'lctl set_param -P' correctly ========================================================== 17:19:00 (1787433540) [ 1372.906759] Lustre: DEBUG MARKER: oleg211-server.virtnet: executing load_modules_local [ 1383.338249] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1388.667898] Lustre: DEBUG MARKER: oleg211-server.virtnet: executing set_default_debug -1 all [ 1396.956098] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1402.097933] Lustre: DEBUG MARKER: oleg211-server.virtnet: executing set_default_debug -1 all [ 1405.197678] Lustre: 48841:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify general/debug_raw_pointers=Y (mode = 0) failed: rc = -17 [ 1411.927848] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 1418.707465] Lustre: DEBUG MARKER: oleg211-server.virtnet: executing set_default_debug -1 all [ 1427.448846] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 1436.100560] Lustre: DEBUG MARKER: oleg211-server.virtnet: executing set_default_debug -1 all [ 1444.295356] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 1448.830418] Lustre: Setting parameter general.llite.*.xattr_cache=0 in log params [ 1462.123157] Lustre: server umount lustre-MDT0000 complete [ 1463.787377] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 1466.231587] LustreError: 47670:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1787433654 with bad export cookie 2640131335954338933 [ 1466.235797] LustreError: MGC192.168.202.111@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 1466.248945] LustreError: 47670:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 6 previous similar messages [ 1466.706215] Lustre: server umount lustre-MDT0001 complete [ 1481.697987] Lustre: server umount lustre-OST0000 complete [ 1485.776116] Lustre: 3627:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1787433658/real 1787433658] req@ffff9c92ff092a00 x1874258633914240/t0(0) o400->lustre-MDT0001-lwp-OST0001@0@lo:12/10 lens 224/224 e 0 to 1 dl 1787433674 ref 1 fl Rpc:XNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 [ 1486.342548] Lustre: server umount lustre-OST0001 complete [ 1496.843573] Lustre: DEBUG MARKER: == conf-sanity test 77: comma-separated MGS NIDs and failover node NIDs ========================================================== 17:21:23 (1787433683) [ 1499.338978] Lustre: DEBUG MARKER: SKIP: conf-sanity test_77 mixed loopback and real device not working [ 1501.619171] Lustre: DEBUG MARKER: == conf-sanity test 78: run resize2fs on MDT and OST filesystems ========================================================== 17:21:27 (1787433687) [ 1510.772812] Lustre: DEBUG MARKER: oleg211-server.virtnet: executing set_hostid [ 1519.162953] Lustre: DEBUG MARKER: oleg211-server.virtnet: executing load_modules_local [ 1540.338198] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1540.567571] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 1540.715931] Lustre: lustre-MDT0000: new disk, initializing [ 1540.800424] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 1544.540372] Lustre: DEBUG MARKER: oleg211-server.virtnet: executing set_default_debug -1 all [ 1553.090645] Lustre: DEBUG MARKER: oleg211-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 1559.393907] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 1559.688275] Lustre: lustre-OST0000: new disk, initializing [ 1559.695687] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 1559.701818] Lustre: Skipped 1 previous similar message [ 1559.707916] Lustre: 54914:0:(osd_compat.c:1353:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 1561.437618] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:0:ost [ 1561.451829] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:0:ost] [ 1561.495054] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x240000400 [ 1565.187079] Lustre: DEBUG MARKER: oleg211-server.virtnet: executing set_default_debug -1 all [ 1573.378819] Lustre: DEBUG MARKER: oleg211-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 1575.326503] Lustre: DEBUG MARKER: create test files [ 1602.536940] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 1602.552942] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 1602.555436] Lustre: Skipped 12 previous similar messages [ 1602.561889] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 1602.561898] Lustre: Skipped 18 previous similar messages [ 1607.718199] Lustre: server umount lustre-OST0000 complete [ 1611.252317] Lustre: server umount lustre-MDT0000 complete [ 1622.526406] Lustre: DEBUG MARKER: oleg211-server.virtnet: executing unload_modules_local [ 1624.552294] Key type lgssc unregistered [ 1624.815532] LNet: 56546:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 1624.820109] LNetError: 56546:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 1624.839957] LNet: Removed LNI 192.168.202.111@tcp [ 1625.615157] Key type .llcrypt unregistered [ 1625.619036] Key type ._llcrypt unregistered [ 1649.225929] Key type ._llcrypt registered [ 1649.227644] Key type .llcrypt registered [ 1649.319307] Lustre: DEBUG MARKER: oleg211-server.virtnet: executing load_modules_local [ 1650.682404] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 1650.706101] alg: No test for adler32 (adler32-zlib) [ 1651.721846] Lustre: Lustre: Build Version: 2.17.57_63_gaeb0ce0 [ 1651.901907] LNet: Added LNI 192.168.202.111@tcp [8/256/0/180] [ 1653.527192] Key type lgssc registered [ 1654.600441] Lustre: Echo OBD driver; http://www.lustre.org/ [ 1661.329532] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 1661.364576] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1662.761501] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 1666.413214] Lustre: DEBUG MARKER: oleg211-server.virtnet: executing set_default_debug -1 all [ 1671.941736] Lustre: DEBUG MARKER: oleg211-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 1678.550515] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 1678.990898] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 1683.791958] Lustre: DEBUG MARKER: oleg211-server.virtnet: executing set_default_debug -1 all [ 1687.153454] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x240000400:100 to 0x240000400:129) [ 1687.247268] Lustre: 59259:0:(osd_internal.h:1460:osd_trans_exec_op()) lustre-OST0000: opcode 8: before 510 < left 544, rollback = 9 [ 1687.266023] Lustre: 59259:0:(osd_handler.c:2090:osd_trans_dump_creds()) create: 32/128/5, destroy: 0/0/0 [ 1687.279775] Lustre: 59259:0:(osd_handler.c:2097:osd_trans_dump_creds()) attr_set: 0/0/0, xattr_set: 0/0/0 [ 1687.298609] Lustre: 59259:0:(osd_handler.c:2107:osd_trans_dump_creds()) write: 2/18/1, punch: 0/0/0, quota 1/3/0 [ 1687.306357] Lustre: 59259:0:(osd_handler.c:2114:osd_trans_dump_creds()) insert: 32/544/0, delete: 0/0/0 [ 1687.317895] Lustre: 59259:0:(osd_handler.c:2121:osd_trans_dump_creds()) ref_add: 0/0/0, ref_del: 0/0/0 [ 1690.034726] Lustre: DEBUG MARKER: oleg211-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 1694.597526] Lustre: DEBUG MARKER: check files after expanding the MDT and OST filesystems [ 1698.492459] Lustre: DEBUG MARKER: create more files after expanding the MDT and OST filesystems [ 1707.660549] Lustre: server umount lustre-OST0000 complete [ 1722.510677] Lustre: server umount lustre-MDT0000 complete [ 1731.848471] Lustre: DEBUG MARKER: oleg211-server.virtnet: executing unload_modules_local [ 1734.761432] Key type lgssc unregistered [ 1735.048361] LNet: 60870:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 1735.062662] LNetError: 60870:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 1736.103453] LNet: Removed LNI 192.168.202.111@tcp [ 1736.802729] Key type .llcrypt unregistered [ 1736.805878] Key type ._llcrypt unregistered [ 1761.451161] Key type ._llcrypt registered [ 1761.452699] Key type .llcrypt registered [ 1761.538205] Lustre: DEBUG MARKER: oleg211-server.virtnet: executing load_modules_local [ 1762.448735] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 1762.473714] alg: No test for adler32 (adler32-zlib) [ 1763.444545] Lustre: Lustre: Build Version: 2.17.57_63_gaeb0ce0 [ 1763.618124] LNet: Added LNI 192.168.202.111@tcp [8/256/0/180] [ 1765.255333] Key type lgssc registered [ 1766.083418] Lustre: Echo OBD driver; http://www.lustre.org/ [ 1772.205655] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 1772.220752] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1773.516869] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 1777.289628] Lustre: DEBUG MARKER: oleg211-server.virtnet: executing set_default_debug -1 all [ 1783.607683] Lustre: DEBUG MARKER: oleg211-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 1789.229651] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 1789.471706] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 1795.193699] Lustre: DEBUG MARKER: oleg211-server.virtnet: executing set_default_debug -1 all [ 1799.733225] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x240000400:140 to 0x240000400:161) [ 1799.820440] Lustre: 63684:0:(osd_internal.h:1460:osd_trans_exec_op()) lustre-OST0000: opcode 8: before 509 < left 544, rollback = 9 [ 1799.833210] Lustre: 63684:0:(osd_handler.c:2090:osd_trans_dump_creds()) create: 32/128/6, destroy: 0/0/0 [ 1799.841452] Lustre: 63684:0:(osd_handler.c:2097:osd_trans_dump_creds()) attr_set: 0/0/0, xattr_set: 0/0/0 [ 1799.846724] Lustre: 63684:0:(osd_handler.c:2107:osd_trans_dump_creds()) write: 2/18/1, punch: 0/0/0, quota 1/3/0 [ 1799.857476] Lustre: 63684:0:(osd_handler.c:2114:osd_trans_dump_creds()) insert: 32/544/0, delete: 0/0/0 [ 1799.869157] Lustre: 63684:0:(osd_handler.c:2121:osd_trans_dump_creds()) ref_add: 0/0/0, ref_del: 0/0/0 [ 1802.012690] Lustre: DEBUG MARKER: oleg211-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 1806.492801] Lustre: DEBUG MARKER: check files after shrinking the MDT and OST filesystems [ 1815.011119] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 1815.037744] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 1815.041381] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 1818.982914] Lustre: server umount lustre-OST0000 complete [ 1822.976304] Lustre: server umount lustre-MDT0000 complete [ 1833.439437] Lustre: DEBUG MARKER: oleg211-server.virtnet: executing unload_modules_local [ 1836.580660] Key type lgssc unregistered [ 1836.897884] LNet: 65246:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 1836.911065] LNetError: 65246:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 1836.929177] LNet: Removed LNI 192.168.202.111@tcp [ 1838.111950] Key type .llcrypt unregistered [ 1838.116132] Key type ._llcrypt unregistered [ 1858.204858] Key type ._llcrypt registered [ 1858.209924] Key type .llcrypt registered [ 1858.316210] Lustre: DEBUG MARKER: oleg211-server.virtnet: executing set_hostid [ 1874.162445] Lustre: DEBUG MARKER: oleg211-server.virtnet: executing load_modules_local [ 1875.371844] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 1875.423044] alg: No test for adler32 (adler32-zlib) [ 1876.605225] Lustre: Lustre: Build Version: 2.17.57_63_gaeb0ce0 [ 1877.061315] LNet: Added LNI 192.168.202.111@tcp [8/256/0/180] [ 1878.799663] Key type lgssc registered [ 1880.418888] Lustre: Echo OBD driver; http://www.lustre.org/ [ 1910.808001] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 1910.835533] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1912.072471] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 1912.102334] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 1912.162423] Lustre: lustre-MDT0000: new disk, initializing [ 1912.230781] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 1912.243086] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 1915.980580] Lustre: DEBUG MARKER: oleg211-server.virtnet: executing set_default_debug -1 all [ 1924.944321] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1925.095990] Lustre: 68395:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 1925.217422] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [ 1925.226494] Lustre: Skipped 1 previous similar message [ 1925.310406] Lustre: lustre-MDT0001: new disk, initializing [ 1925.437580] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 1925.471216] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [ 1925.480594] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 1929.397340] Lustre: DEBUG MARKER: oleg211-server.virtnet: executing set_default_debug -1 all [ 1939.357161] Lustre: DEBUG MARKER: oleg211-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 1945.042620] Lustre: DEBUG MARKER: oleg211-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 1951.558154] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 1951.968481] Lustre: lustre-OST0000: new disk, initializing [ 1951.971988] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 1951.978511] Lustre: 70362:0:(osd_compat.c:1353:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 1952.101882] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 1957.449480] Lustre: DEBUG MARKER: oleg211-server.virtnet: executing set_default_debug -1 all [ 1961.519727] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [ 1961.533627] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 1961.646193] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 1966.607591] Lustre: DEBUG MARKER: oleg211-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 1972.094801] Lustre: DEBUG MARKER: oleg211-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 [ 1972.268328] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [ 1977.395069] Lustre: DEBUG MARKER: oleg211-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid 50 [ 1977.678336] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [ 1982.432350] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 1982.433228] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 1982.440988] LustreError: Skipped 1 previous similar message [ 1982.443032] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 1982.462269] Lustre: Skipped 1 previous similar message [ 1985.690653] Lustre: server umount lustre-OST0000 complete [ 1992.162107] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 1992.178984] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 1992.194651] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 1992.205885] Lustre: Skipped 1 previous similar message [ 1996.114788] Lustre: server umount lustre-MDT0000 complete [ 1999.600971] LustreError: 70361:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1787434187 with bad export cookie 16644127251335905183 [ 1999.616714] LustreError: MGC192.168.202.111@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 1999.843202] Lustre: server umount lustre-MDT0001 complete [ 2008.424563] Lustre: DEBUG MARKER: == conf-sanity test 79: format MDT/OST without mgs option (should return errors) ========================================================== 17:29:54 (1787434194) [ 2019.299936] Lustre: DEBUG MARKER: oleg211-server.virtnet: executing load_modules_local [ 2058.013084] Lustre: DEBUG MARKER: oleg211-server.virtnet: executing set_hostid [ 2070.933837] Lustre: DEBUG MARKER: oleg211-server.virtnet: executing load_modules_local [ 2106.076079] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 2106.341814] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 2106.371802] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 2106.430402] Lustre: lustre-MDT0000: new disk, initializing [ 2106.537683] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 2106.565950] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 2111.164532] Lustre: DEBUG MARKER: oleg211-server.virtnet: executing set_default_debug -1 all [ 2121.818071] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 2121.938748] Lustre: 76381:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 2121.996246] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [ 2122.002424] Lustre: Skipped 1 previous similar message [ 2122.078335] Lustre: lustre-MDT0001: new disk, initializing [ 2122.221155] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 2122.259100] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [ 2122.274928] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 2126.690317] Lustre: DEBUG MARKER: oleg211-server.virtnet: executing set_default_debug -1 all [ 2135.939677] Lustre: DEBUG MARKER: oleg211-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 2141.017602] Lustre: DEBUG MARKER: oleg211-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 2147.957229] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 2148.399519] Lustre: lustre-OST0000: new disk, initializing [ 2148.407895] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 2148.421621] Lustre: 78352:0:(osd_compat.c:1353:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 2148.554767] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 2155.504559] Lustre: DEBUG MARKER: oleg211-server.virtnet: executing set_default_debug -1 all [ 2158.658946] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [ 2158.680769] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 2158.908426] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 2165.883062] Lustre: DEBUG MARKER: oleg211-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 2170.533884] Lustre: DEBUG MARKER: oleg211-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 [ 2170.729476] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [ 2176.711935] Lustre: DEBUG MARKER: oleg211-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid 50 [ 2177.159818] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [ 2179.552630] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 2179.553891] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2179.558615] LustreError: Skipped 1 previous similar message [ 2179.560143] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 2179.560162] Lustre: Skipped 1 previous similar message [ 2179.615410] Lustre: Skipped 2 previous similar messages [ 2184.672972] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 2184.698839] Lustre: Skipped 2 previous similar messages [ 2185.662398] Lustre: server umount lustre-OST0000 complete [ 2189.795032] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2189.805702] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 2189.806798] Lustre: Skipped 1 previous similar message [ 2189.817367] Lustre: Skipped 1 previous similar message [ 2195.948122] Lustre: server umount lustre-MDT0000 complete [ 2199.773775] LustreError: 76375:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1787434388 with bad export cookie 16644127251335906072 [ 2199.776696] LustreError: MGC192.168.202.111@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 2199.792133] LustreError: 76375:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 7 previous similar messages [ 2200.146747] Lustre: server umount lustre-MDT0001 complete [ 2209.527928] Lustre: DEBUG MARKER: == conf-sanity test 80: mgc import reconnect race ======== 17:33:15 (1787434395) [ 2217.950803] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 2218.505538] LustreError: 80764:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 2218.537212] LustreError: 80764:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 3 previous similar messages [ 2218.646923] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 2224.099930] LustreError: 80765:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 2225.071828] Lustre: DEBUG MARKER: oleg211-server.virtnet: executing set_default_debug -1 all [ 2228.195433] LustreError: 80764:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 2233.320648] LustreError: 80765:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 2235.975652] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 2242.068219] Lustre: DEBUG MARKER: oleg211-server.virtnet: executing set_default_debug -1 all [ 2249.328693] Lustre: DEBUG MARKER: oleg211-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 2255.623728] Lustre: DEBUG MARKER: oleg211-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 2262.361088] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 2262.718151] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 2262.729409] Lustre: Skipped 1 previous similar message [ 2269.357772] Lustre: DEBUG MARKER: oleg211-server.virtnet: executing set_default_debug -1 all [ 2276.288626] Lustre: DEBUG MARKER: oleg211-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 2280.248727] Lustre: 83446:0:(genops.c:1774:obd_export_evict_by_uuid()) MGS: evicting 65a90e9c-fec8-4df2-9d00-a75dd9e54690 at adminstrative request [ 2281.058660] LustreError: MGC192.168.202.111@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 2281.078716] Lustre: *** cfs_fail_loc=906, val=2147483648*** [ 2291.688685] Lustre: MGS: Client 65a90e9c-fec8-4df2-9d00-a75dd9e54690 (at 0@lo) reconnecting [ 2291.706348] Lustre: Evicted from MGS (at 0@lo) after server handle changed from 0xe6fbd18cfc46868a to 0xe6fbd18cfc46892a [ 2291.743682] Lustre: MGC192.168.202.111@tcp: Connection restored to 0@lo (at 0@lo) [ 2317.307654] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 2317.415837] Lustre: lustre-OST0001: new disk, initializing [ 2317.418624] Lustre: srv-lustre-OST0001: No data found on store. Initialize space. [ 2317.423733] Lustre: 83839:0:(osd_compat.c:1353:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 2323.603202] Lustre: DEBUG MARKER: oleg211-server.virtnet: executing set_default_debug -1 all [ 2324.122740] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x00000002c0000400-0x0000000300000400]:1:ost [ 2324.128345] Lustre: cli-lustre-OST0001-super: Allocated super-sequence [0x00000002c0000400-0x0000000300000400]:1:ost] [ 2324.200773] Lustre: lustre-OST0001-osc-MDT0000: update sequence from 0x100010000 to 0x2c0000401 [ 2334.789734] Lustre: DEBUG MARKER: oleg211-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0001-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 2339.813605] LustreError: lustre-OST0001-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 2339.832411] Lustre: lustre-OST0001-osc-MDT0001: Connection to lustre-OST0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2339.833930] Lustre: lustre-OST0001: Not available for connect from 0@lo (stopping) [ 2339.862679] Lustre: Skipped 1 previous similar message [ 2339.880364] Lustre: Skipped 3 previous similar messages [ 2344.013480] Lustre: server umount lustre-OST0001 complete [ 2344.936339] LustreError: 82593:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 2344.963128] LustreError: 82593:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 1 previous similar message [ 2350.053547] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2350.078805] Lustre: Skipped 1 previous similar message [ 2355.169390] LustreError: 82593:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 2355.191557] LustreError: 82593:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 3 previous similar messages [ 2355.425376] Lustre: server umount lustre-OST0000 complete [ 2360.297647] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 2360.306119] Lustre: Skipped 4 previous similar messages [ 2366.927338] Lustre: server umount lustre-MDT0000 complete [ 2372.328947] LustreError: 83838:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1787434560 with bad export cookie 16644127251335907626 [ 2372.332356] LustreError: MGC192.168.202.111@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 2372.347047] LustreError: 83838:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 2 previous similar messages [ 2373.222694] Lustre: server umount lustre-MDT0001 complete [ 2382.146759] Lustre: DEBUG MARKER: == conf-sanity test 81: sparse OST indexing ============== 17:36:08 (1787434568) [ 2383.448438] Lustre: DEBUG MARKER: SKIP: conf-sanity test_81 needs >= 3 OSTs [ 2384.921675] Lustre: DEBUG MARKER: == conf-sanity test 82a: specify OSTs for file (succeed) or directory (succeed) ========================================================== 17:36:11 (1787434571) [ 2386.226679] Lustre: DEBUG MARKER: SKIP: conf-sanity test_82a needs >= 3 OSTs [ 2387.720903] Lustre: DEBUG MARKER: == conf-sanity test 82b: specify OSTs for file with --pool and --ost-list options ========================================================== 17:36:14 (1787434574) [ 2389.125982] Lustre: DEBUG MARKER: SKIP: conf-sanity test_82b needs >= 4 OSTs [ 2390.699512] Lustre: DEBUG MARKER: == conf-sanity test 82c: specify sparse OSTs for setstripe (should not crash) ========================================================== 17:36:17 (1787434577) [ 2392.388984] Lustre: DEBUG MARKER: SKIP: conf-sanity test_82c needs >= 4 OSTs [ 2393.864898] Lustre: DEBUG MARKER: == conf-sanity test 83: ENOSPACE on OST doesn't cause message VFS: Busy inodes after unmount ... ========================================================== 17:36:20 (1787434580) [ 2395.296089] Lustre: DEBUG MARKER: mount the OST /dev/mapper/ost1_flakey as a ldiskfs filesystem [ 2400.888073] loop: module loaded [ 2400.935309] loop0: detected capacity change from 0 to 4294967296 [ 2401.011581] LDISKFS-fs (loop0): mounted filesystem with ordered data mode. Opts: (null) [ 2402.643677] Lustre: DEBUG MARKER: run llverfs in partial mode on the OST ldiskfs /mnt/lustre-ost1 [ 2406.444329] Lustre: DEBUG MARKER: oleg211-server.virtnet: executing run_llverfs /mnt/lustre-ost1 -vpl no [ 2411.930147] Lustre: DEBUG MARKER: unmount the OST /dev/mapper/ost1_flakey [ 2441.734603] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 2441.780127] LustreError: 87966:0:(obd_config.c:845:class_setup()) setup lustre-OST0000-osd failed (-28) [ 2441.791412] LustreError: 87966:0:(obd_mount.c:259:lustre_start_simple()) lustre-OST0000-osd setup error -28 [ 2441.798271] LustreError: 87966:0:(tgt_mount.c:2579:server_fill_super()) Unable to start osd on /dev/mapper/ost1_flakey: -28 [ 2441.803812] LustreError: 87966:0:(super25.c:179:lustre_fill_super()) llite: Unable to mount : rc = -28 [ 2461.919911] Lustre: DEBUG MARKER: oleg211-server.virtnet: executing set_hostid [ 2471.260373] Lustre: DEBUG MARKER: oleg211-server.virtnet: executing load_modules_local [ 2504.975842] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 2505.169975] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 2505.189970] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 2505.251880] Lustre: lustre-MDT0000: new disk, initializing [ 2505.316311] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 2505.320570] Lustre: Skipped 1 previous similar message [ 2505.335727] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 2510.290724] Lustre: DEBUG MARKER: oleg211-server.virtnet: executing set_default_debug -1 all [ 2521.162354] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 2521.246253] Lustre: 90831:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 2521.482888] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 2529.292138] Lustre: DEBUG MARKER: oleg211-server.virtnet: executing set_default_debug -1 all [ 2540.447683] Lustre: DEBUG MARKER: oleg211-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 2546.276656] Lustre: DEBUG MARKER: oleg211-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 2553.300170] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 2553.861397] Lustre: 92798:0:(osd_compat.c:1353:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 2559.575850] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 2559.772951] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 2560.237697] Lustre: DEBUG MARKER: oleg211-server.virtnet: executing set_default_debug -1 all [ 2569.352871] Lustre: DEBUG MARKER: oleg211-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 2573.683273] Lustre: DEBUG MARKER: oleg211-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 [ 2573.828246] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [ 2578.856124] Lustre: DEBUG MARKER: oleg211-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid 50 [ 2579.207425] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [ 2585.571862] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 2585.572403] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2585.575672] LustreError: Skipped 2 previous similar messages [ 2585.576959] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 2585.600672] Lustre: Skipped 3 previous similar messages [ 2585.614320] Lustre: Skipped 3 previous similar messages [ 2587.916928] Lustre: server umount lustre-OST0000 complete [ 2599.678693] Lustre: server umount lustre-MDT0000 complete [ 2600.930084] LustreError: 90842:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 2600.984800] LustreError: 90842:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 3 previous similar messages [ 2604.454301] LustreError: 90824:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1787434792 with bad export cookie 16644127251335908242 [ 2604.458146] LustreError: MGC192.168.202.111@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 2604.467114] LustreError: 90824:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 2604.787845] Lustre: server umount lustre-MDT0001 complete [ 2614.835803] Lustre: DEBUG MARKER: == conf-sanity test 84: check recovery_hard_time ========= 17:40:00 (1787434800) [ 2621.528555] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 2627.064555] Lustre: DEBUG MARKER: oleg211-server.virtnet: executing set_default_debug -1 all [ 2635.695142] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 2636.037936] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-60 [ 2636.042509] Lustre: Skipped 3 previous similar messages [ 2641.217780] Lustre: DEBUG MARKER: oleg211-server.virtnet: executing set_default_debug -1 all [ 2649.275670] Lustre: DEBUG MARKER: oleg211-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 2655.009910] Lustre: DEBUG MARKER: oleg211-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 2662.424103] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 2670.308534] Lustre: DEBUG MARKER: oleg211-server.virtnet: executing set_default_debug -1 all [ 2679.364061] Lustre: DEBUG MARKER: oleg211-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 2684.089285] Lustre: DEBUG MARKER: oleg211-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 [ 2684.276540] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [ 2691.371584] Lustre: DEBUG MARKER: oleg211-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid 50 [ 2691.740406] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [ 2698.961280] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 2699.139756] Lustre: lustre-OST0001: new disk, initializing [ 2699.143112] Lustre: Skipped 2 previous similar messages [ 2699.146704] Lustre: srv-lustre-OST0001: No data found on store. Initialize space. [ 2699.150154] Lustre: Skipped 3 previous similar messages [ 2699.155316] Lustre: 98584:0:(osd_compat.c:1353:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 2706.057867] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x00000002c0000400-0x0000000300000400]:1:ost [ 2706.069681] Lustre: Skipped 2 previous similar messages [ 2706.087713] Lustre: cli-lustre-OST0001-super: Allocated super-sequence [0x00000002c0000400-0x0000000300000400]:1:ost] [ 2706.209874] Lustre: lustre-OST0001-osc-MDT0000: update sequence from 0x100010000 to 0x2c0000401 [ 2706.293124] Lustre: DEBUG MARKER: oleg211-server.virtnet: executing set_default_debug -1 all [ 2716.333448] Lustre: DEBUG MARKER: oleg211-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0001-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 2721.837219] Lustre: DEBUG MARKER: oleg211-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0001-osc-MDT0000.ost_server_uuid 50 [ 2722.020086] Lustre: DEBUG MARKER: os[cp].lustre-OST0001-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [ 2726.876571] Lustre: DEBUG MARKER: oleg211-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0001-osc-MDT0001.ost_server_uuid 50 [ 2727.109595] Lustre: DEBUG MARKER: os[cp].lustre-OST0001-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [ 2745.996764] Lustre: DEBUG MARKER: mds1 REPLAY BARRIER on lustre-MDT0000 [ 2766.918472] Lustre: Failing over lustre-MDT0000 [ 2767.456651] Lustre: server umount lustre-MDT0000 complete [ 2767.846364] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2767.853123] LustreError: 95208:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 2767.862667] Lustre: Skipped 5 previous similar messages [ 2767.880523] LustreError: 95208:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 6 previous similar messages [ 2784.736269] Lustre: 66623:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1787434956/real 1787434956] req@00000000a62f1db3 x1874260462413824/t0(0) o400->MGC192.168.202.111@tcp@0@lo:26/25 lens 224/224 e 0 to 1 dl 1787434972 ref 1 fl Rpc:XNQr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 [ 2784.781425] LustreError: MGC192.168.202.111@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 2790.762733] LDISKFS-fs (dm-0): 4 truncates cleaned up [ 2790.765351] LDISKFS-fs (dm-0): recovery complete [ 2790.771848] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 2794.988098] Lustre: Evicted from MGS (at 0@lo) after server handle changed from 0x0 to 0xe6fbd18cfc47539e [ 2795.011449] Lustre: MGC192.168.202.111@tcp: Connection restored to 0@lo (at 0@lo) [ 2795.323871] Lustre: lustre-MDT0000: in recovery but waiting for the first client to connect [ 2797.051325] Lustre: lustre-MDT0000: Will be in recovery for at least 1:00, or until 3 clients reconnect [ 2797.056671] Lustre: 95209:0:(ldlm_lib.c:2110:extend_recovery_timer()) lustre-MDT0000: extended recovery timer reached hard limit: 60, extend: 0 [ 2799.171720] Lustre: DEBUG MARKER: oleg211-server.virtnet: executing set_default_debug -1 all [ 2800.612283] Lustre: 95941:0:(ldlm_lib.c:2110:extend_recovery_timer()) lustre-MDT0000: extended recovery timer reached hard limit: 60, extend: 0 [ 2800.622568] Lustre: lustre-MDT0000-lwp-MDT0001: Connection restored to 0@lo (at 0@lo) [ 2800.626362] Lustre: 95941:0:(ldlm_lib.c:2110:extend_recovery_timer()) Skipped 1 previous similar message [ 2800.702977] LustreError: 101431:0:(ldlm_lib.c:2723:replay_request_or_update()) cfs_fail_timeout id 709 sleeping for 300ms [ 2801.039458] LustreError: 101431:0:(ldlm_lib.c:2723:replay_request_or_update()) cfs_fail_timeout id 709 awake [ 2801.362354] LustreError: 101431:0:(ldlm_lib.c:2723:replay_request_or_update()) cfs_fail_timeout id 709 sleeping for 300ms [ 2801.389022] LustreError: 101431:0:(ldlm_lib.c:2723:replay_request_or_update()) Skipped 1 previous similar message [ 2801.695134] LustreError: 101431:0:(ldlm_lib.c:2723:replay_request_or_update()) cfs_fail_timeout id 709 awake [ 2801.702475] LustreError: 101431:0:(ldlm_lib.c:2723:replay_request_or_update()) Skipped 1 previous similar message [ 2801.714665] Lustre: 101431:0:(ldlm_lib.c:2110:extend_recovery_timer()) lustre-MDT0000: extended recovery timer reached hard limit: 60, extend: 1 [ 2801.723776] Lustre: 101431:0:(ldlm_lib.c:2110:extend_recovery_timer()) Skipped 6 previous similar messages [ 2802.668734] LustreError: 101431:0:(ldlm_lib.c:2723:replay_request_or_update()) cfs_fail_timeout id 709 sleeping for 300ms [ 2802.700685] LustreError: 101431:0:(ldlm_lib.c:2723:replay_request_or_update()) Skipped 3 previous similar messages [ 2803.045802] LustreError: 101431:0:(ldlm_lib.c:2723:replay_request_or_update()) cfs_fail_timeout id 709 awake [ 2803.061367] LustreError: 101431:0:(ldlm_lib.c:2723:replay_request_or_update()) Skipped 3 previous similar messages [ 2803.728199] Lustre: 101431:0:(ldlm_lib.c:2110:extend_recovery_timer()) lustre-MDT0000: extended recovery timer reached hard limit: 60, extend: 1 [ 2803.741423] Lustre: 101431:0:(ldlm_lib.c:2110:extend_recovery_timer()) Skipped 5 previous similar messages [ 2804.700702] LustreError: 101431:0:(ldlm_lib.c:2723:replay_request_or_update()) cfs_fail_timeout id 709 sleeping for 300ms [ 2804.709610] LustreError: 101431:0:(ldlm_lib.c:2723:replay_request_or_update()) Skipped 5 previous similar messages [ 2805.343190] LustreError: 101431:0:(ldlm_lib.c:2723:replay_request_or_update()) cfs_fail_timeout id 709 awake [ 2805.356387] LustreError: 101431:0:(ldlm_lib.c:2723:replay_request_or_update()) Skipped 6 previous similar messages [ 2807.933784] Lustre: 101431:0:(ldlm_lib.c:2110:extend_recovery_timer()) lustre-MDT0000: extended recovery timer reached hard limit: 60, extend: 1 [ 2807.958325] Lustre: 101431:0:(ldlm_lib.c:2110:extend_recovery_timer()) Skipped 12 previous similar messages [ 2808.929244] LustreError: 101431:0:(ldlm_lib.c:2723:replay_request_or_update()) cfs_fail_timeout id 709 sleeping for 300ms [ 2808.944917] LustreError: 101431:0:(ldlm_lib.c:2723:replay_request_or_update()) Skipped 12 previous similar messages [ 2809.609480] LustreError: 101431:0:(ldlm_lib.c:2723:replay_request_or_update()) cfs_fail_timeout id 709 awake [ 2809.622238] LustreError: 101431:0:(ldlm_lib.c:2723:replay_request_or_update()) Skipped 12 previous similar messages [ 2816.089344] Lustre: 101431:0:(ldlm_lib.c:2110:extend_recovery_timer()) lustre-MDT0000: extended recovery timer reached hard limit: 60, extend: 1 [ 2816.099598] Lustre: 101431:0:(ldlm_lib.c:2110:extend_recovery_timer()) Skipped 24 previous similar messages [ 2817.053899] LustreError: 101431:0:(ldlm_lib.c:2723:replay_request_or_update()) cfs_fail_timeout id 709 sleeping for 300ms [ 2817.079133] LustreError: 101431:0:(ldlm_lib.c:2723:replay_request_or_update()) Skipped 24 previous similar messages [ 2817.712744] LustreError: 101431:0:(ldlm_lib.c:2723:replay_request_or_update()) cfs_fail_timeout id 709 awake [ 2817.735314] LustreError: 101431:0:(ldlm_lib.c:2723:replay_request_or_update()) Skipped 24 previous similar messages [ 2832.316781] Lustre: 101431:0:(ldlm_lib.c:2110:extend_recovery_timer()) lustre-MDT0000: extended recovery timer reached hard limit: 60, extend: 1 [ 2832.323387] Lustre: 101431:0:(ldlm_lib.c:2110:extend_recovery_timer()) Skipped 50 previous similar messages [ 2833.289980] LustreError: 101431:0:(ldlm_lib.c:2723:replay_request_or_update()) cfs_fail_timeout id 709 sleeping for 300ms [ 2833.310762] LustreError: 101431:0:(ldlm_lib.c:2723:replay_request_or_update()) Skipped 50 previous similar messages [ 2833.975921] LustreError: 101431:0:(ldlm_lib.c:2723:replay_request_or_update()) cfs_fail_timeout id 709 awake [ 2833.989670] LustreError: 101431:0:(ldlm_lib.c:2723:replay_request_or_update()) Skipped 50 previous similar messages [ 2857.208437] Lustre: 101431:0:(ldlm_lib.c:2419:target_recovery_overseer()) lustre-MDT0000 recovery is aborted by hard timeout [ 2857.219623] Lustre: 101431:0:(ldlm_lib.c:2429:target_recovery_overseer()) recovery is aborted, evict exports in recovery [ 2857.231195] Lustre: 101431:0:(genops.c:1601:class_disconnect_stale_exports()) lustre-MDT0000: disconnect stale client d9497ba0-dce9-439f-a170-b985ac06ff60@192.168.202.11@tcp [ 2857.245060] Lustre: lustre-MDT0000: disconnecting 1 stale clients [ 2857.254554] Lustre: 101431:0:(ldlm_lib.c:1939:abort_req_replay_queue()) @@@ aborted: req@000000001a3c35f1 x1874260444384256/t0(8589934773) o101->d9497ba0-dce9-439f-a170-b985ac06ff60@192.168.202.11@tcp:491/0 lens 592/0 e 0 to 0 dl 1787435056 ref 1 fl Complete:/604/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:0 [ 2857.282269] Lustre: lustre-MDT0000-osd: cancel update llog [0x200000400:0x1:0x0] [ 2857.304850] Lustre: lustre-MDT0000: Denying connection for new client d9497ba0-dce9-439f-a170-b985ac06ff60 (at 192.168.202.11@tcp), waiting for 3 known clients (2 recovered, 0 in progress, and 1 evicted) to recover in 0:00 [ 2857.327516] Lustre: lustre-MDT0001-osp-MDT0000: cancel update llog [0x240000401:0x1:0x0] [ 2857.361084] Lustre: lustre-MDT0000-osp-MDT0001: Connection restored to 0@lo (at 0@lo) [ 2857.365725] Lustre: lustre-MDT0000: Recovery over after 1:00, of 3 clients 2 recovered and 1 was evicted. [ 2857.366763] Lustre: Skipped 2 previous similar messages [ 2857.471074] Lustre: lustre-OST0001: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x2c0000401:46 to 0x2c0000401:545) [ 2857.472464] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:78 to 0x280000401:577) [ 2870.752211] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 2870.758090] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2870.772323] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 2870.778474] Lustre: Skipped 2 previous similar messages [ 2876.511443] Lustre: server umount lustre-OST0000 complete [ 2877.425060] LustreError: 98019:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-OST0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 2877.439671] LustreError: 98019:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 41 previous similar messages [ 2885.772730] Lustre: server umount lustre-OST0001 complete [ 2895.726887] Lustre: server umount lustre-MDT0000 complete [ 2899.665362] LustreError: 95949:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1787435087 with bad export cookie 16644127251335959454 [ 2899.680629] LustreError: MGC192.168.202.111@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 2900.207887] Lustre: server umount lustre-MDT0001 complete [ 2909.927959] Lustre: DEBUG MARKER: == conf-sanity test 85: osd_ost init: fail ea_fid_set ==== 17:44:56 (1787435096) [ 2918.024030] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 2918.036884] Lustre: *** cfs_fail_loc=197, val=0*** [ 2933.663644] LustreError: Failed to get MGS log lustre-sptlrpc and no local copy. [ 2938.784268] LustreError: 103404:0:(mgc_request.c:233:do_config_log_add()) MGC192.168.202.111@tcp: failed processing log, type 4: rc = -110 [ 2943.968771] Lustre: *** cfs_fail_loc=197, val=0*** [ 2943.976958] Lustre: Skipped 65 previous similar messages [ 2964.575489] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 2964.581794] Lustre: Skipped 3 previous similar messages [ 2970.529661] Lustre: DEBUG MARKER: oleg211-server.virtnet: executing set_default_debug -1 all [ 2978.298931] Lustre: DEBUG MARKER: oleg211-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 2980.685324] Lustre: server umount lustre-OST0000 complete [ 2989.343752] Lustre: DEBUG MARKER: == conf-sanity test 86: Replacing mkfs.lustre -G option == 17:46:15 (1787435175) [ 3010.951739] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 3011.398436] LustreError: 105026:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 3011.421516] LustreError: 105026:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 4 previous similar messages [ 3017.100694] Lustre: DEBUG MARKER: oleg211-server.virtnet: executing set_default_debug -1 all [ 3026.584493] Lustre: DEBUG MARKER: oleg211-client.virtnet: executing wait_import_state_mount (FULL|IDLE) mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 3030.436524] LustreError: 105057:0:(osp_object.c:618:osp_attr_get()) lustre-MDT0001-osp-MDT0000: osp_attr_get update error [0x200000009:0x1:0x0]: rc = -5 [ 3030.453846] LustreError: 105057:0:(lod_sub_object.c:917:lod_sub_prep_llog()) lustre-MDT0000-mdtlov: can't get id from catalogs: rc = -5 [ 3030.461783] LustreError: 105057:0:(lod_dev.c:511:lod_sub_recovery_thread()) lustre-MDT0001-osp-MDT0000: get update log duration 19, retries 0, failed: rc = -5 [ 3030.839990] Lustre: server umount lustre-MDT0000 complete [ 3064.055822] Lustre: DEBUG MARKER: oleg211-server.virtnet: executing set_hostid [ 3073.328527] Lustre: DEBUG MARKER: oleg211-server.virtnet: executing load_modules_local [ 3109.941453] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 3110.180044] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 3110.200189] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 3110.266880] Lustre: lustre-MDT0000: new disk, initializing [ 3110.335891] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 3114.811297] Lustre: DEBUG MARKER: oleg211-server.virtnet: executing set_default_debug -1 all [ 3126.741388] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 3126.911439] Lustre: 109229:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 3127.084295] Lustre: lustre-MDT0001: Not available for connect from 0@lo (not set up) [ 3127.091961] Lustre: Skipped 7 previous similar messages [ 3127.226952] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 3132.888298] Lustre: DEBUG MARKER: oleg211-server.virtnet: executing set_default_debug -1 all [ 3144.623271] Lustre: DEBUG MARKER: oleg211-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 3150.155812] Lustre: DEBUG MARKER: oleg211-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 3157.230343] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 3157.461784] Lustre: 111199:0:(osd_compat.c:1353:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 3162.769386] Lustre: DEBUG MARKER: oleg211-server.virtnet: executing set_default_debug -1 all [ 3165.890959] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 3171.280883] Lustre: DEBUG MARKER: oleg211-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 3175.869834] Lustre: DEBUG MARKER: oleg211-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 [ 3176.076263] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [ 3180.576945] Lustre: DEBUG MARKER: oleg211-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid 50 [ 3180.739784] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [ 3186.670496] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 3186.670611] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 3186.683452] LustreError: Skipped 2 previous similar messages [ 3186.716083] Lustre: Skipped 6 previous similar messages [ 3188.769749] Lustre: server umount lustre-OST0000 complete [ 3198.447125] Lustre: server umount lustre-MDT0000 complete [ 3202.701760] LustreError: 109224:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1787435390 with bad export cookie 16644127251335970080 [ 3202.702557] LustreError: MGC192.168.202.111@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 3202.708895] LustreError: 109224:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 3 previous similar messages [ 3203.308364] Lustre: server umount lustre-MDT0001 complete [ 3212.047559] Lustre: DEBUG MARKER: == conf-sanity test 87: check if MDT inode can hold EAs with N stripes properly ========================================================== 17:49:58 (1787435398) [ 3221.277777] Lustre: DEBUG MARKER: oleg211-server.virtnet: executing unload_modules_local [ 3223.927309] Key type lgssc unregistered [ 3224.253971] LNet: 113689:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 3224.259393] LNetError: 113689:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 3224.278500] LNet: Removed LNI 192.168.202.111@tcp [ 3225.117472] Key type .llcrypt unregistered [ 3225.119520] Key type ._llcrypt unregistered [ 3247.058811] Key type ._llcrypt registered [ 3247.061890] Key type .llcrypt registered [ 3247.213599] Lustre: DEBUG MARKER: oleg211-server.virtnet: executing set_hostid [ 3268.866307] Lustre: DEBUG MARKER: oleg211-server.virtnet: executing load_modules_local [ 3270.446030] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 3270.656233] alg: No test for adler32 (adler32-zlib) [ 3271.940963] Lustre: Lustre: Build Version: 2.17.57_63_gaeb0ce0 [ 3272.144639] LNet: Added LNI 192.168.202.111@tcp [8/256/0/180] [ 3273.882814] Key type lgssc registered [ 3274.738862] Lustre: Echo OBD driver; http://www.lustre.org/ [ 3321.393165] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 3321.439038] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 3322.902076] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 3322.985878] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 3323.089166] Lustre: lustre-MDT0000: new disk, initializing [ 3323.173317] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 3323.198806] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 3329.363252] Lustre: DEBUG MARKER: oleg211-server.virtnet: executing set_default_debug -1 all [ 3341.596965] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 3341.944857] Lustre: lustre-OST0000: new disk, initializing [ 3341.948208] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 3341.955368] Lustre: Skipped 1 previous similar message [ 3341.963781] Lustre: 117869:0:(osd_compat.c:1353:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 3342.061185] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 3348.007147] Lustre: DEBUG MARKER: oleg211-server.virtnet: executing set_default_debug -1 all [ 3349.153233] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:0:ost [ 3349.164187] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:0:ost] [ 3349.211820] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x240000400 [ 3359.141526] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 3359.314360] Lustre: lustre-OST0001: new disk, initializing [ 3359.317088] Lustre: srv-lustre-OST0001: No data found on store. Initialize space. [ 3359.323690] Lustre: 118732:0:(osd_compat.c:1353:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 3359.371834] Lustre: lustre-OST0001: Imperative Recovery not enabled, recovery window 60-180 [ 3365.303360] Lustre: DEBUG MARKER: oleg211-server.virtnet: executing set_default_debug -1 all [ 3368.656701] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:1:ost [ 3368.678615] Lustre: cli-lustre-OST0001-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:1:ost] [ 3368.713485] Lustre: lustre-OST0001-osc-MDT0000: update sequence from 0x100010000 to 0x280000400 [ 3379.171291] Lustre: lustre-MDT0000-lwp-OST0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 3379.199957] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 3380.715900] Lustre: server umount lustre-MDT0000 complete [ 3387.820812] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: (null) [ 3392.441376] LustreError: 117057:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1787435580 with bad export cookie 2520814993828010434 [ 3392.451510] LustreError: MGC192.168.202.111@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 3392.593899] Lustre: server umount lustre-OST0000 complete [ 3395.940253] Lustre: server umount lustre-OST0001 complete [ 3405.103945] Lustre: DEBUG MARKER: == conf-sanity test 88a: tunefs.lustre persists a --mountfsoptions-only change ========================================================== 17:53:11 (1787435591) [ 3427.641636] Lustre: DEBUG MARKER: == conf-sanity test 88b: check the default mount options can be overridden ========================================================== 17:53:33 (1787435613) [ 3438.128434] Lustre: DEBUG MARKER: oleg211-server.virtnet: executing load_modules_local [ 3464.230525] Lustre: DEBUG MARKER: == conf-sanity test 89: check tunefs --param and --erase-param{s} options ========================================================== 17:54:10 (1787435650) [ 3506.201371] Lustre: DEBUG MARKER: oleg211-server.virtnet: executing set_hostid [ 3518.779599] Lustre: DEBUG MARKER: oleg211-server.virtnet: executing load_modules_local [ 3561.660299] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 3561.828427] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 3561.869307] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 3562.018372] Lustre: lustre-MDT0000: new disk, initializing [ 3562.118303] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 3562.158582] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 3567.115666] Lustre: DEBUG MARKER: oleg211-server.virtnet: executing set_default_debug -1 all [ 3579.501537] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 3579.697633] Lustre: 126504:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 3579.771451] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [ 3579.776584] Lustre: Skipped 1 previous similar message [ 3579.913347] Lustre: lustre-MDT0001: new disk, initializing [ 3580.122078] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 3580.235109] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [ 3580.262111] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 3585.699738] Lustre: DEBUG MARKER: oleg211-server.virtnet: executing set_default_debug -1 all [ 3595.218459] Lustre: DEBUG MARKER: oleg211-client.virtnet: executing wait_import_state_mount FULL failover.node=192.0.2.254@tcp0,mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 3601.379183] Lustre: DEBUG MARKER: oleg211-client.virtnet: executing wait_import_state_mount FULL failover.node=192.0.2.254@tcp0,mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 3608.766179] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 3609.041066] Lustre: lustre-OST0000: new disk, initializing [ 3609.044261] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 3609.048862] Lustre: 128474:0:(osd_compat.c:1353:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 3609.176701] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 3615.950759] Lustre: DEBUG MARKER: oleg211-server.virtnet: executing set_default_debug -1 all [ 3616.892393] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [ 3616.906479] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 3617.079397] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 3626.486798] Lustre: DEBUG MARKER: oleg211-client.virtnet: executing wait_import_state_mount (FULL|IDLE) failover.node=192.0.2.254@tcp0,osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 3631.566726] Lustre: DEBUG MARKER: oleg211-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 [ 3631.820924] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [ 3637.414284] Lustre: DEBUG MARKER: oleg211-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid 50 [ 3637.682172] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [ 3642.857529] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 3642.879045] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 3642.883187] Lustre: Skipped 2 previous similar messages [ 3642.895323] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 3642.897478] Lustre: Skipped 1 previous similar message [ 3642.926199] LustreError: Skipped 1 previous similar message [ 3645.839613] Lustre: server umount lustre-OST0000 complete [ 3651.552321] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 3651.558610] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 3651.579709] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 3651.588338] Lustre: Skipped 1 previous similar message [ 3655.661664] Lustre: server umount lustre-MDT0000 complete [ 3659.498120] LustreError: 130219:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1787435847 with bad export cookie 2520814993828011659 [ 3659.498553] LustreError: MGC192.168.202.111@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 3659.504716] LustreError: 130219:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 1 previous similar message [ 3659.926190] Lustre: server umount lustre-MDT0001 complete [ 3669.660882] Lustre: DEBUG MARKER: == conf-sanity test 90a: check max_mod_rpcs_in_flight is enforced ========================================================== 17:57:35 (1787435855) [ 3677.317554] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 3678.091652] LustreError: 130887:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 3678.115193] LustreError: 130887:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 3 previous similar messages [ 3678.206582] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 3683.305395] LustreError: 130888:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 3684.172494] Lustre: DEBUG MARKER: oleg211-server.virtnet: executing set_default_debug -1 all [ 3688.426620] LustreError: 130887:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 3692.604855] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 3697.486806] Lustre: DEBUG MARKER: oleg211-server.virtnet: executing set_default_debug -1 all [ 3707.471691] Lustre: DEBUG MARKER: oleg211-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 3714.553284] Lustre: DEBUG MARKER: oleg211-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 3724.294827] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 3725.033312] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 3725.043855] Lustre: Skipped 1 previous similar message [ 3734.154849] Lustre: DEBUG MARKER: oleg211-server.virtnet: executing set_default_debug -1 all [ 3742.653524] Lustre: DEBUG MARKER: oleg211-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 3757.806796] Lustre: *** cfs_fail_loc=159, val=0*** [ 3757.810982] Lustre: Skipped 2 previous similar messages [ 3773.498640] Lustre: lustre-MDT0000: Client 33e75043-55b3-4963-a112-c3b70d9d9346 (at 192.168.202.11@tcp) reconnecting [ 3775.967968] Lustre: *** cfs_fail_loc=159, val=0*** [ 3775.976126] Lustre: Skipped 3 previous similar messages [ 3791.439323] Lustre: lustre-MDT0000: Client 33e75043-55b3-4963-a112-c3b70d9d9346 (at 192.168.202.11@tcp) reconnecting [ 3799.035147] Lustre: server umount lustre-OST0000 complete [ 3805.155928] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 3805.172120] Lustre: Skipped 1 previous similar message [ 3805.173707] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 3805.187111] Lustre: Skipped 3 previous similar messages [ 3810.279396] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 3810.285046] Lustre: Skipped 1 previous similar message [ 3813.475848] Lustre: server umount lustre-MDT0000 complete [ 3815.398379] LustreError: 131897:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 3815.428076] LustreError: 131897:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 1 previous similar message [ 3816.351242] Lustre: 115075:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1787435988/real 1787435988] req@00000000a9708ab2 x1874261925016576/t0(0) o13->lustre-OST0000-osc-MDT0001@0@lo:7/4 lens 224/368 e 0 to 1 dl 1787436004 ref 1 fl Rpc:XQr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 [ 3816.383803] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 3817.606416] LustreError: 131622:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1787436005 with bad export cookie 2520814993828012541 [ 3817.613734] LustreError: MGC192.168.202.111@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 3817.614417] LustreError: 131622:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 3817.928892] Lustre: server umount lustre-MDT0001 complete [ 3828.372710] Lustre: DEBUG MARKER: oleg211-server.virtnet: executing unload_modules_local [ 3831.330538] Key type lgssc unregistered [ 3831.608662] LNet: 134690:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 3831.615412] LNetError: 134690:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 3831.629913] LNet: Removed LNI 192.168.202.111@tcp [ 3832.582968] Key type .llcrypt unregistered [ 3832.585710] Key type ._llcrypt unregistered [ 3857.071558] Key type ._llcrypt registered [ 3857.074025] Key type .llcrypt registered [ 3857.211545] Lustre: DEBUG MARKER: oleg211-server.virtnet: executing load_modules_local [ 3858.780943] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 3858.824719] alg: No test for adler32 (adler32-zlib) [ 3859.988591] Lustre: Lustre: Build Version: 2.17.57_63_gaeb0ce0 [ 3860.366315] LNet: Added LNI 192.168.202.111@tcp [8/256/0/180] [ 3862.071543] Key type lgssc registered [ 3863.735117] Lustre: Echo OBD driver; http://www.lustre.org/ [ 3874.248763] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 3874.305488] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 3876.059969] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 3883.301314] Lustre: DEBUG MARKER: oleg211-server.virtnet: executing set_default_debug -1 all [ 3886.563942] LustreError: 136118:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 3886.598217] LustreError: 136118:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 2 previous similar messages [ 3891.683555] LustreError: 136117:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 3891.926541] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 3892.552635] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 3898.337814] Lustre: DEBUG MARKER: oleg211-server.virtnet: executing set_default_debug -1 all [ 3906.090950] Lustre: DEBUG MARKER: oleg211-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 3914.261730] Lustre: DEBUG MARKER: oleg211-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 3923.196877] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 3923.778210] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 3933.103788] Lustre: DEBUG MARKER: oleg211-server.virtnet: executing set_default_debug -1 all [ 3942.645704] Lustre: DEBUG MARKER: oleg211-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 3946.319184] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:42 to 0x280000401:65) [ 3956.358647] Lustre: *** cfs_fail_loc=159, val=0*** [ 3972.685399] Lustre: lustre-MDT0000: Client 538830f8-89e5-41e9-9edc-de479eccef7f (at 192.168.202.11@tcp) reconnecting [ 3975.414506] Lustre: *** cfs_fail_loc=159, val=0*** [ 3991.647653] Lustre: lustre-MDT0001: Client 538830f8-89e5-41e9-9edc-de479eccef7f (at 192.168.202.11@tcp) reconnecting [ 3994.372325] Lustre: *** cfs_fail_loc=159, val=0*** [ 3994.374056] Lustre: Skipped 3 previous similar messages [ 4010.562978] Lustre: lustre-MDT0001: Client 538830f8-89e5-41e9-9edc-de479eccef7f (at 192.168.202.11@tcp) reconnecting [ 4015.212204] Lustre: 139157:0:(mdt_handler.c:64:mdt_max_mod_rpcs_per_client_set()) max_mod_rpcs_per_client is deprecated, set mdt.*.max_mod_rpcs_in_flight parameter instead [ 4018.342910] Lustre: *** cfs_fail_loc=159, val=0*** [ 4018.352534] Lustre: Skipped 4 previous similar messages [ 4034.600600] Lustre: lustre-MDT0000: Client 882bee57-db84-4287-ae24-5fbfe68f3650 (at 192.168.202.11@tcp) reconnecting [ 4036.201921] Lustre: *** cfs_fail_loc=159, val=0*** [ 4036.206681] Lustre: Skipped 14 previous similar messages [ 4052.552683] Lustre: lustre-MDT0000: Client 882bee57-db84-4287-ae24-5fbfe68f3650 (at 192.168.202.11@tcp) reconnecting [ 4054.368342] Lustre: 139445:0:(mdt_handler.c:64:mdt_max_mod_rpcs_per_client_set()) max_mod_rpcs_per_client is deprecated, set mdt.*.max_mod_rpcs_in_flight parameter instead [ 4064.225438] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4064.230023] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 4064.282104] Lustre: Skipped 1 previous similar message [ 4064.317953] Lustre: Skipped 1 previous similar message [ 4066.855235] Lustre: server umount lustre-OST0000 complete [ 4071.888164] Lustre: server umount lustre-MDT0000 complete [ 4071.913559] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 4071.923128] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4071.937150] LustreError: 138993:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 4074.468573] LustreError: 136118:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 4074.485430] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4075.953994] LustreError: 136099:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1787436264 with bad export cookie 16689268669896647821 [ 4075.958226] LustreError: MGC192.168.202.111@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 4075.963315] LustreError: 136099:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 2 previous similar messages [ 4076.239622] Lustre: server umount lustre-MDT0001 complete [ 4089.188338] Lustre: DEBUG MARKER: oleg211-server.virtnet: executing unload_modules_local [ 4092.285590] Key type lgssc unregistered [ 4092.475860] LNet: 140500:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 4092.483426] LNetError: 140500:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 4093.546264] LNet: Removed LNI 192.168.202.111@tcp [ 4094.663902] Key type .llcrypt unregistered [ 4094.674412] Key type ._llcrypt unregistered [ 4125.424492] Key type ._llcrypt registered [ 4125.427252] Key type .llcrypt registered [ 4125.548424] Lustre: DEBUG MARKER: oleg211-server.virtnet: executing load_modules_local [ 4127.223579] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 4127.393097] alg: No test for adler32 (adler32-zlib) [ 4128.585945] Lustre: Lustre: Build Version: 2.17.57_63_gaeb0ce0 [ 4128.836245] LNet: Added LNI 192.168.202.111@tcp [8/256/0/180] [ 4130.511181] Key type lgssc registered [ 4132.168117] Lustre: Echo OBD driver; http://www.lustre.org/ [ 4140.890672] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 4140.917249] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 4142.549990] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 4148.292201] Lustre: DEBUG MARKER: oleg211-server.virtnet: executing set_default_debug -1 all [ 4151.782400] LustreError: 141918:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 4151.824258] LustreError: 141918:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 2 previous similar messages [ 4156.915996] LustreError: 141917:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 4159.037184] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 4159.441924] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 4164.233736] Lustre: DEBUG MARKER: oleg211-server.virtnet: executing set_default_debug -1 all [ 4174.564243] Lustre: DEBUG MARKER: oleg211-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 4180.383650] Lustre: DEBUG MARKER: oleg211-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 4187.453712] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 4187.772769] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 4194.135917] Lustre: DEBUG MARKER: oleg211-server.virtnet: executing set_default_debug -1 all [ 4203.984437] Lustre: DEBUG MARKER: oleg211-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 4204.173298] Lustre: lustre-OST0000: new connection from lustre-MDT0001-mdtlov (cleaning up unused objects from 0x280000400:72 to 0x280000400:97) [ 4212.367191] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:85 to 0x280000401:129) [ 4219.461795] Lustre: 144525:0:(mdt_handler.c:64:mdt_max_mod_rpcs_per_client_set()) max_mod_rpcs_per_client is deprecated, set mdt.*.max_mod_rpcs_in_flight parameter instead [ 4222.647647] Lustre: 144573:0:(mdt_handler.c:64:mdt_max_mod_rpcs_per_client_set()) max_mod_rpcs_per_client is deprecated, set mdt.*.max_mod_rpcs_in_flight parameter instead [ 4224.529187] Lustre: Modifying parameter lustre.mdc.lustre-MDT0000-mdc-*.max_rpcs_in_flight=8 in log params [ 4225.561086] Lustre: Setting parameter lustre.mdc.lustre-MDT0000-mdc-*.max_mod_rpcs_in_flight=8 in log params [ 4230.111242] Lustre: 144720:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre/mdc.lustre-MDT0000-mdc-*.max_rpcs_in_flight=8 (mode = 0) failed: rc = -17 [ 4231.199936] Lustre: Modifying parameter lustre.mdc.lustre-MDT0000-mdc-*.max_mod_rpcs_in_flight=7 in log params [ 4232.261791] Lustre: 144819:0:(mdt_handler.c:64:mdt_max_mod_rpcs_per_client_set()) max_mod_rpcs_per_client is deprecated, set mdt.*.max_mod_rpcs_in_flight parameter instead [ 4238.308477] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 4238.319686] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4238.343861] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 4238.346324] Lustre: Skipped 1 previous similar message [ 4238.375374] Lustre: Skipped 1 previous similar message [ 4243.427132] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 4243.443730] Lustre: Skipped 1 previous similar message [ 4244.561657] Lustre: server umount lustre-OST0000 complete [ 4251.626146] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 4251.643343] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4251.678836] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 4253.666049] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4253.689127] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 4253.691801] Lustre: Skipped 1 previous similar message [ 4256.230731] Lustre: server umount lustre-MDT0000 complete [ 4258.785130] LustreError: 143080:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 4258.797938] LustreError: 143080:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 1 previous similar message [ 4259.895315] LustreError: 141899:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1787436448 with bad export cookie 16246557568829440128 [ 4259.906113] LustreError: MGC192.168.202.111@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 4259.908993] LustreError: 141899:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 2 previous similar messages [ 4260.341570] Lustre: server umount lustre-MDT0001 complete [ 4272.281495] Lustre: DEBUG MARKER: oleg211-server.virtnet: executing unload_modules_local [ 4275.519405] Key type lgssc unregistered [ 4275.915610] LNet: 145882:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 4275.923915] LNetError: 145882:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 4275.942193] LNet: Removed LNI 192.168.202.111@tcp [ 4276.825266] Key type .llcrypt unregistered [ 4276.827033] Key type ._llcrypt unregistered [ 4308.864147] Key type ._llcrypt registered [ 4308.869028] Key type .llcrypt registered [ 4309.008668] Lustre: DEBUG MARKER: oleg211-server.virtnet: executing load_modules_local [ 4310.347930] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 4310.524086] alg: No test for adler32 (adler32-zlib) [ 4311.775593] Lustre: Lustre: Build Version: 2.17.57_63_gaeb0ce0 [ 4312.211646] LNet: Added LNI 192.168.202.111@tcp [8/256/0/180] [ 4314.031195] Key type lgssc registered [ 4315.453594] Lustre: Echo OBD driver; http://www.lustre.org/ [ 4324.673392] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 4324.723707] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 4326.342718] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 4331.674569] Lustre: DEBUG MARKER: oleg211-server.virtnet: executing set_default_debug -1 all [ 4336.610373] LustreError: 147300:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 4336.629045] LustreError: 147300:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 2 previous similar messages [ 4341.736636] LustreError: 147299:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 4341.969988] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 4342.464708] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 4346.865737] Lustre: DEBUG MARKER: oleg211-server.virtnet: executing set_default_debug -1 all [ 4355.053870] Lustre: DEBUG MARKER: oleg211-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 4360.861951] Lustre: DEBUG MARKER: oleg211-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 4368.107041] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 4368.639844] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 4371.738158] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:85 to 0x280000401:161) [ 4375.228574] Lustre: DEBUG MARKER: oleg211-server.virtnet: executing set_default_debug -1 all [ 4383.067714] Lustre: DEBUG MARKER: oleg211-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 4387.054495] Lustre: lustre-OST0000: new connection from lustre-MDT0001-mdtlov (cleaning up unused objects from 0x280000400:72 to 0x280000400:129) [ 4392.864608] Lustre: *** cfs_fail_loc=159, val=0*** [ 4408.395237] Lustre: lustre-MDT0000: Client ff7574f1-9544-48a9-9e84-cec6734e4d06 (at 192.168.202.11@tcp) reconnecting [ 4428.256509] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 4428.261907] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4428.266660] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 4428.300122] Lustre: Skipped 1 previous similar message [ 4432.524571] Lustre: server umount lustre-OST0000 complete [ 4438.499299] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4438.507478] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 4438.521735] Lustre: Skipped 1 previous similar message [ 4438.549992] Lustre: Skipped 2 previous similar messages [ 4442.845421] Lustre: server umount lustre-MDT0000 complete [ 4443.617483] LustreError: 149817:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 4443.650976] LustreError: 149817:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 1 previous similar message [ 4447.326702] LustreError: 147280:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1787436635 with bad export cookie 17766629942625666745 [ 4447.329324] LustreError: MGC192.168.202.111@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 4447.341690] LustreError: 147280:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 1 previous similar message [ 4448.144157] Lustre: server umount lustre-MDT0001 complete [ 4464.099704] Lustre: DEBUG MARKER: oleg211-server.virtnet: executing unload_modules_local [ 4469.404469] Key type lgssc unregistered [ 4469.919661] LNet: 150973:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 4469.934726] LNetError: 150973:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 4469.983815] LNet: Removed LNI 192.168.202.111@tcp [ 4471.466482] Key type .llcrypt unregistered [ 4471.471718] Key type ._llcrypt unregistered [ 4501.962900] Key type ._llcrypt registered [ 4501.966422] Key type .llcrypt registered [ 4502.119337] Lustre: DEBUG MARKER: oleg211-server.virtnet: executing load_modules_local [ 4503.336026] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 4503.481709] alg: No test for adler32 (adler32-zlib) [ 4504.716090] Lustre: Lustre: Build Version: 2.17.57_63_gaeb0ce0 [ 4505.271671] LNet: Added LNI 192.168.202.111@tcp [8/256/0/180] [ 4507.135891] Key type lgssc registered [ 4509.206224] Lustre: Echo OBD driver; http://www.lustre.org/ [ 4518.886386] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 4518.900361] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 4520.318330] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 4524.490620] Lustre: DEBUG MARKER: oleg211-server.virtnet: executing set_default_debug -1 all [ 4530.664413] LustreError: 152389:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 4530.678875] LustreError: 152389:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 2 previous similar messages [ 4533.465552] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 4533.862525] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 4539.375403] Lustre: DEBUG MARKER: oleg211-server.virtnet: executing set_default_debug -1 all [ 4546.690411] Lustre: DEBUG MARKER: oleg211-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 4552.411296] Lustre: DEBUG MARKER: oleg211-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 4560.580918] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 4561.174467] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 4565.253780] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:170 to 0x280000401:193) [ 4570.829843] Lustre: DEBUG MARKER: oleg211-server.virtnet: executing set_default_debug -1 all [ 4578.614108] Lustre: lustre-OST0000: new connection from lustre-MDT0001-mdtlov (cleaning up unused objects from 0x280000400:72 to 0x280000400:161) [ 4583.192742] Lustre: DEBUG MARKER: oleg211-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 4603.284141] Lustre: lustre-MDT0001: evicting f50147ca-5a79-4000-b43c-af8e6e55f7f4 (at 192.168.202.11@tcp) by administrative request [ 4603.286684] Lustre: Skipped 1 previous similar message [ 4632.032859] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 4632.041569] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4632.059295] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 4635.104305] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 4635.105600] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 4635.117782] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4635.135863] Lustre: Skipped 1 previous similar message [ 4640.225996] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 4645.360974] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 4645.372563] Lustre: Skipped 2 previous similar messages [ 4646.368988] Lustre: lustre-OST0000 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 4646.582923] Lustre: server umount lustre-OST0000 complete [ 4650.472136] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4650.484809] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 4650.496252] Lustre: Skipped 1 previous similar message [ 4656.719674] Lustre: server umount lustre-MDT0000 complete [ 4660.704837] LustreError: 152390:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 4660.730494] LustreError: 152390:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 1 previous similar message [ 4662.221902] LustreError: 152370:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1787436850 with bad export cookie 11667428684857860106 [ 4662.225961] LustreError: MGC192.168.202.111@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 4662.241925] LustreError: 152370:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 4663.073615] Lustre: server umount lustre-MDT0001 complete [ 4677.991105] Lustre: DEBUG MARKER: oleg211-server.virtnet: executing unload_modules_local [ 4681.278868] Key type lgssc unregistered [ 4681.540103] LNet: 156462:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 4681.553737] LNetError: 156462:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 4682.600479] LNet: Removed LNI 192.168.202.111@tcp [ 4683.914102] Key type .llcrypt unregistered [ 4683.918700] Key type ._llcrypt unregistered [ 4768.372172] Key type ._llcrypt registered [ 4768.378825] Key type .llcrypt registered [ 4768.549588] Lustre: DEBUG MARKER: oleg211-server.virtnet: executing load_modules_local [ 4770.515594] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 4770.592273] alg: No test for adler32 (adler32-zlib) [ 4771.742345] Lustre: Lustre: Build Version: 2.17.57_63_gaeb0ce0 [ 4771.992273] LNet: Added LNI 192.168.202.111@tcp [8/256/0/180] [ 4773.680957] Key type lgssc registered [ 4775.717820] Lustre: Echo OBD driver; http://www.lustre.org/ [ 4787.298654] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 4787.330853] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 4789.174355] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 4794.659504] Lustre: DEBUG MARKER: oleg211-server.virtnet: executing set_default_debug -1 all [ 4799.458891] LustreError: 158957:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 4799.475989] LustreError: 158957:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 2 previous similar messages [ 4803.106101] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 4803.417751] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 4808.520958] Lustre: DEBUG MARKER: oleg211-server.virtnet: executing set_default_debug -1 all [ 4818.235324] Lustre: DEBUG MARKER: oleg211-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 4825.853137] Lustre: DEBUG MARKER: oleg211-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 4834.309764] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 4834.820959] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 4842.596459] Lustre: DEBUG MARKER: oleg211-server.virtnet: executing set_default_debug -1 all [ 4848.122636] Lustre: lustre-OST0000: new connection from lustre-MDT0001-mdtlov (cleaning up unused objects from 0x280000400:72 to 0x280000400:193) [ 4850.276963] Lustre: DEBUG MARKER: oleg211-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 4859.382503] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:195 to 0x280000401:225) [ 4869.605150] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4869.610587] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 4869.650508] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 4874.730124] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 4874.735543] Lustre: Skipped 1 previous similar message [ 4875.322617] Lustre: server umount lustre-OST0000 complete [ 4884.960933] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4884.979786] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 4884.993672] Lustre: Skipped 2 previous similar messages [ 4885.034262] Lustre: Skipped 2 previous similar messages [ 4889.062393] Lustre: server umount lustre-MDT0000 complete [ 4890.088873] LustreError: 158957:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 4890.106498] LustreError: 158957:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 1 previous similar message [ 4894.508652] LustreError: 158938:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1787437082 with bad export cookie 17351539710018136184 [ 4894.520062] LustreError: MGC192.168.202.111@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 4894.822936] Lustre: server umount lustre-MDT0001 complete [ 4906.750340] Lustre: DEBUG MARKER: oleg211-server.virtnet: executing unload_modules_local [ 4909.645779] Key type lgssc unregistered [ 4910.201705] LNet: 162522:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 4910.223737] LNetError: 162522:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 4910.262859] LNet: Removed LNI 192.168.202.111@tcp [ 4911.777947] Key type .llcrypt unregistered [ 4911.786745] Key type ._llcrypt unregistered [ 4937.681715] Key type ._llcrypt registered [ 4937.684239] Key type .llcrypt registered [ 4937.800455] Lustre: DEBUG MARKER: oleg211-server.virtnet: executing load_modules_local [ 4939.060471] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 4939.118575] alg: No test for adler32 (adler32-zlib) [ 4940.287648] Lustre: Lustre: Build Version: 2.17.57_63_gaeb0ce0 [ 4940.559696] LNet: Added LNI 192.168.202.111@tcp [8/256/0/180] [ 4942.327157] Key type lgssc registered [ 4943.736891] Lustre: Echo OBD driver; http://www.lustre.org/ [ 4960.058818] Lustre: DEBUG MARKER: == conf-sanity test 100: check lshowmount lists MGS, MDT, OST and 0@lo ========================================================== 18:19:05 (1787437145) [ 4979.226969] Lustre: DEBUG MARKER: oleg211-server.virtnet: executing set_hostid [ 4990.324426] Lustre: DEBUG MARKER: oleg211-server.virtnet: executing load_modules_local [ 5030.332630] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 5030.377720] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 5031.748023] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 5031.852144] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 5031.989192] Lustre: lustre-MDT0000: new disk, initializing [ 5032.153894] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 5032.215605] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 5037.005780] Lustre: DEBUG MARKER: oleg211-server.virtnet: executing set_default_debug -1 all [ 5047.424734] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 5047.566228] Lustre: 166792:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 5047.604041] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [ 5047.607596] Lustre: Skipped 1 previous similar message [ 5047.667515] Lustre: lustre-MDT0001: new disk, initializing [ 5047.743936] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 5047.774910] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [ 5047.786814] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 5052.165159] Lustre: DEBUG MARKER: oleg211-server.virtnet: executing set_default_debug -1 all [ 5061.090159] Lustre: DEBUG MARKER: oleg211-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 5066.839568] Lustre: DEBUG MARKER: oleg211-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 5073.354601] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 5073.624584] Lustre: lustre-OST0000: new disk, initializing [ 5073.627986] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 5073.634115] Lustre: 168760:0:(osd_compat.c:1353:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 5073.721350] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 5079.323496] Lustre: DEBUG MARKER: oleg211-server.virtnet: executing set_default_debug -1 all [ 5081.649509] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [ 5081.663597] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 5081.803244] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 5090.344456] Lustre: DEBUG MARKER: oleg211-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 5107.680370] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 5107.684576] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5107.691195] LustreError: Skipped 1 previous similar message [ 5107.691968] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 5107.735957] Lustre: Skipped 1 previous similar message [ 5109.120083] Lustre: server umount lustre-OST0000 complete [ 5114.339613] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 5114.349242] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5114.368598] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 5114.371270] Lustre: Skipped 1 previous similar message [ 5117.921585] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 5117.922604] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5123.041024] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 5123.046594] Lustre: Skipped 2 previous similar messages [ 5128.163367] Lustre: lustre-MDT0000 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 5128.166723] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 5128.197201] Lustre: Skipped 1 previous similar message [ 5128.408343] Lustre: server umount lustre-MDT0000 complete [ 5131.937248] LustreError: 166784:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1787437320 with bad export cookie 11568876652234282762 [ 5131.951132] LustreError: 166784:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 2 previous similar messages [ 5131.951511] LustreError: MGC192.168.202.111@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 5132.379816] Lustre: server umount lustre-MDT0001 complete [ 5146.831864] Lustre: DEBUG MARKER: oleg211-server.virtnet: executing unload_modules_local [ 5149.951978] Key type lgssc unregistered [ 5150.240569] LNet: 170671:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 5150.260316] LNetError: 170671:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 5150.308616] LNet: Removed LNI 192.168.202.111@tcp [ 5151.637716] Key type .llcrypt unregistered [ 5151.640418] Key type ._llcrypt unregistered [ 5180.255789] Key type ._llcrypt registered [ 5180.261086] Key type .llcrypt registered [ 5180.396377] Lustre: DEBUG MARKER: oleg211-server.virtnet: executing load_modules_local [ 5182.073977] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 5182.121085] alg: No test for adler32 (adler32-zlib) [ 5183.579283] Lustre: Lustre: Build Version: 2.17.57_63_gaeb0ce0 [ 5184.225612] LNet: Added LNI 192.168.202.111@tcp [8/256/0/180] [ 5185.943234] Key type lgssc registered [ 5187.528916] Lustre: Echo OBD driver; http://www.lustre.org/ [ 5199.418472] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 5199.464134] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 5200.912240] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 5205.840968] Lustre: DEBUG MARKER: oleg211-server.virtnet: executing set_default_debug -1 all [ 5211.110575] LustreError: 172104:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 5211.138502] LustreError: 172104:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 2 previous similar messages [ 5216.231849] LustreError: 172103:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 5216.318355] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 5216.809335] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 5221.129348] Lustre: DEBUG MARKER: oleg211-server.virtnet: executing set_default_debug -1 all [ 5228.263693] Lustre: DEBUG MARKER: oleg211-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 5234.755756] Lustre: DEBUG MARKER: oleg211-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 5244.164229] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 5244.742481] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 5246.790290] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:3 to 0x280000401:33) [ 5255.204511] Lustre: DEBUG MARKER: oleg211-server.virtnet: executing set_default_debug -1 all [ 5265.406191] Lustre: DEBUG MARKER: oleg211-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 5274.507029] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 5274.542489] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5274.589342] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 5274.611248] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5274.634984] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 5276.260452] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 5276.277522] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5276.284354] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 5276.289317] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5276.301748] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 5277.277502] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 5277.303307] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5277.321957] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 5277.329993] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5277.339260] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 5278.393211] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5279.328833] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 5279.333987] Lustre: Skipped 1 previous similar message [ 5279.350995] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5279.359966] Lustre: Skipped 1 previous similar message [ 5279.382120] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 5279.386486] Lustre: Skipped 1 previous similar message [ 5279.393035] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5279.407831] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 5279.414147] Lustre: Skipped 1 previous similar message [ 5280.426868] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5281.307633] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5282.366627] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5283.420139] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 5283.429875] Lustre: Skipped 3 previous similar messages [ 5283.470643] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5283.482406] Lustre: Skipped 3 previous similar messages [ 5283.486685] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 5283.494720] Lustre: Skipped 3 previous similar messages [ 5283.503650] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5283.523503] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 5283.528408] Lustre: Skipped 3 previous similar messages [ 5284.429946] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5285.500568] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5286.627859] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5287.660899] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5288.759095] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5289.659516] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5290.673595] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5291.777365] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 5291.788952] Lustre: Skipped 7 previous similar messages [ 5291.815628] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5291.827866] Lustre: Skipped 7 previous similar messages [ 5291.832096] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 5291.839922] Lustre: Skipped 7 previous similar messages [ 5291.852041] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5291.874789] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 5291.886178] Lustre: Skipped 7 previous similar messages [ 5292.819539] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5293.609179] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5294.416076] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5295.323175] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5296.357119] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5297.352300] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5298.418365] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5299.698219] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5300.845578] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5301.801693] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5303.477375] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5304.742469] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5305.815355] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5306.921747] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5307.924477] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 5307.933561] Lustre: Skipped 14 previous similar messages [ 5307.955909] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5307.964145] Lustre: Skipped 14 previous similar messages [ 5307.967719] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 5307.970834] Lustre: Skipped 14 previous similar messages [ 5307.975449] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5307.985096] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 5307.992262] Lustre: Skipped 14 previous similar messages [ 5308.706441] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5309.628369] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5310.723513] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5312.157329] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5313.903942] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5315.499880] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5316.815860] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5317.826534] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5318.737356] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5319.621404] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5320.615991] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5321.922341] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5322.862558] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5324.651232] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5325.989256] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5327.020674] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5327.949128] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5328.552356] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5329.445347] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5330.585651] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5331.471578] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5332.327915] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5333.417188] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5334.447032] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5335.500045] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5336.314260] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5337.332264] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5338.046376] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5338.896503] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5339.697700] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5340.455948] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 5340.461710] Lustre: Skipped 30 previous similar messages [ 5340.483856] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5340.490435] Lustre: Skipped 30 previous similar messages [ 5340.492702] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 5340.497536] Lustre: Skipped 30 previous similar messages [ 5340.505429] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5340.518194] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 5340.525984] Lustre: Skipped 30 previous similar messages [ 5341.230129] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5342.175602] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5342.999076] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5343.802852] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5344.718212] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5345.541581] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5346.474928] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5347.278537] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5348.094336] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5348.747690] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5349.651598] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5350.633716] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5351.354911] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5352.271826] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5353.338930] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5354.148144] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5355.048888] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5355.880921] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5356.954911] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5357.803684] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5358.704563] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5359.429121] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5360.153426] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5361.024194] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5361.857196] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5362.610289] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5363.332712] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5364.477428] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5365.572545] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5366.476503] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5367.504574] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5368.229332] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5369.172889] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5369.975131] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5371.070307] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5371.932263] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5373.070997] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5373.911019] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5374.786071] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5375.465107] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5376.339421] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5377.085744] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5377.874680] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5378.829989] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5379.779529] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5380.817980] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5381.698783] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5382.711449] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5383.446530] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5384.612912] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5385.719219] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5386.675394] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5387.824276] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5388.748632] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5390.026252] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5390.963885] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5391.934362] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5393.080804] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5394.472715] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5395.663948] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5396.641861] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5397.604316] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5398.306913] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5399.334638] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5400.310041] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5401.305993] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5402.235635] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5403.164137] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5404.057808] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5404.940170] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 5404.944984] Lustre: Skipped 69 previous similar messages [ 5404.956118] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5404.960964] Lustre: Skipped 69 previous similar messages [ 5404.965254] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 5404.969637] Lustre: Skipped 69 previous similar messages [ 5404.972167] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5404.978966] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 5404.990083] Lustre: Skipped 69 previous similar messages [ 5405.902491] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5406.776702] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5407.635676] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5408.533680] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5409.370587] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5410.361624] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5411.078576] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5411.790629] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5412.747257] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5414.107755] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5415.370483] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5416.223475] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5417.011192] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5417.736656] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5418.539942] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5419.421829] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5420.403443] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5421.258677] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5422.141992] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5423.185195] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5424.223626] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5425.218126] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5426.348877] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5427.194966] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5428.049774] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5428.896692] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5429.715666] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5430.503696] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5431.265375] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5432.168092] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5433.184161] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5434.109047] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5435.223711] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5436.230707] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5437.119130] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5438.204074] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5439.060964] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5439.798606] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5440.492856] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5441.468542] LustreError: 172123:0:(osp_precreate.c:655:osp_precreate_send()) lustre-OST0000-osc-MDT0000: can't precreate: rc = -5 [ 5441.485467] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5441.486196] LustreError: 172123:0:(osp_precreate.c:1359:osp_precreate_thread()) lustre-OST0000-osc-MDT0000: cannot precreate objects: rc = -5 [ 5442.367404] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5443.338135] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5444.080906] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5444.920760] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5445.709980] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5446.687688] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5447.596335] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5448.988887] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5450.215881] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5451.246881] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5452.108812] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5453.094573] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5454.002936] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5454.879946] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5456.486880] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5457.769013] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5458.562993] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5459.615869] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5460.318303] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5461.167636] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5461.991828] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5463.238255] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5464.632059] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5466.264215] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5467.542871] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5468.502907] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5469.718509] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5470.830559] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5471.690516] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5472.906556] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5473.924295] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5474.902801] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5476.316217] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5477.368171] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5478.436137] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5479.567791] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5480.188342] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5481.219580] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5482.092445] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5483.187344] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5483.977873] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5484.907883] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5485.786458] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5486.793334] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5487.567518] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5488.403209] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5489.244358] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5489.965273] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5491.296775] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5492.126885] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5493.053487] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5493.887199] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5494.976555] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5495.983372] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5497.033670] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5498.081815] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5499.165705] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5500.164749] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5501.061536] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5502.121188] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5503.023693] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5504.122313] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5504.967135] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5505.873159] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5506.687330] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5507.658213] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5508.503692] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5509.442280] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5510.357543] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5511.559417] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5512.641509] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5513.849146] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5515.194583] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5516.368746] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5517.574380] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5518.303681] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5519.185630] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5520.073995] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5520.898621] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5521.626684] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5522.389837] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5523.043619] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5523.926771] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5524.674472] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5525.293764] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5526.071395] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5527.178104] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5528.115647] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5528.991084] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5529.942045] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5530.711287] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5531.455582] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5532.660681] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5533.573896] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 5533.578198] Lustre: Skipped 133 previous similar messages [ 5533.602460] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5533.611806] Lustre: Skipped 133 previous similar messages [ 5533.619395] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 5533.622104] Lustre: Skipped 133 previous similar messages [ 5533.627113] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5533.640382] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 5533.656328] Lustre: Skipped 133 previous similar messages [ 5534.918391] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5535.981865] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5536.985573] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5537.957797] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5538.843635] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5539.780331] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5540.756738] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5541.768688] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5542.692987] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5543.628993] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5544.511126] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5545.583907] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5546.444257] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5547.368535] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5548.096447] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5548.872239] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5549.769840] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5550.670661] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5551.518627] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5552.436635] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5553.259898] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5554.019587] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5554.869800] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5555.737242] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5556.576289] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5557.374799] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5558.183550] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5558.976976] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5559.908530] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5560.880253] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5561.866076] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5562.661215] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5563.538174] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5564.417742] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5565.243024] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5566.073147] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5566.825297] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5567.663266] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5568.656398] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5569.612655] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5570.649754] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5571.417289] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5572.269143] LustreError: 172123:0:(osp_precreate.c:655:osp_precreate_send()) lustre-OST0000-osc-MDT0000: can't precreate: rc = -5 [ 5572.287284] LustreError: 172123:0:(osp_precreate.c:1359:osp_precreate_thread()) lustre-OST0000-osc-MDT0000: cannot precreate objects: rc = -5 [ 5572.296821] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5573.166978] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5574.204415] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5575.181491] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5575.931852] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5576.888518] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5577.812138] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5578.699094] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5579.617791] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5580.346860] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5581.237618] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5582.090106] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5582.956630] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5583.893620] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5584.880200] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5585.725512] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5586.631773] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5587.480595] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5588.339736] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5589.179842] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5590.125307] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5590.917108] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5591.803289] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5592.726771] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5593.718679] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5594.431931] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5595.276669] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5596.144092] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5597.093270] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5597.939292] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5598.880081] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5599.782711] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5600.771385] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5601.570148] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5602.399060] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5603.083950] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5603.924721] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5604.639619] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5605.484526] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5606.358066] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5607.588821] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5608.529466] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5609.630352] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5610.542872] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5611.424119] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5612.137918] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5612.789265] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5613.513283] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5614.492597] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5615.351726] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5616.330771] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5617.300468] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5618.416368] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5619.466829] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5620.398808] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5621.443247] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5622.306591] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5623.241321] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5624.461634] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5625.181418] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5626.036419] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5626.991221] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5628.170293] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5629.335572] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5630.326175] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5631.572294] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5632.587290] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5633.701721] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5634.907716] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5635.968577] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5636.824493] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5637.898060] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5638.948483] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5639.944136] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5640.888496] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5641.559926] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5642.507657] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5643.432824] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5644.341077] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5645.151636] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5646.038790] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5646.954619] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5647.805345] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5648.484407] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5649.426222] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5650.096375] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5650.938916] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5651.784755] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5652.644822] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5653.428925] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5654.247811] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5655.155874] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5655.947411] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5657.185114] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5658.425635] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5659.625919] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5660.537041] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5661.446808] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5662.223376] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5662.993904] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5663.782859] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5664.723345] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5665.445482] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5666.372700] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5667.030497] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5667.838325] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5668.594496] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5669.415624] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5670.113309] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5671.018941] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5671.888482] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5672.593154] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5673.370355] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5674.093645] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5674.930148] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5675.839097] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5676.644056] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5677.430776] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5678.328883] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5679.051217] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5679.847187] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5680.720151] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5681.601266] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5682.306289] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5683.114154] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5683.897193] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5684.690213] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5685.406135] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5686.167767] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5687.242714] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5688.200448] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5689.280622] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5690.127849] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5691.034756] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5691.930273] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5692.750472] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5693.666920] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5694.548816] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5695.282681] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5696.238199] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5697.079893] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5698.043881] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5699.057712] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5700.187052] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5701.329935] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5702.403795] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5703.402976] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5704.476465] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5705.388489] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5706.477910] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5707.315567] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5708.234786] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5709.149356] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5710.251448] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5711.092831] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5711.894898] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5712.798242] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5713.614695] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5714.488783] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5715.328193] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5716.140217] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5717.091184] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5717.983776] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5718.968473] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5719.821584] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5720.511420] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5721.332938] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5722.089984] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5722.716407] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5723.530266] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5724.245358] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5724.987274] LustreError: 172123:0:(osp_precreate.c:655:osp_precreate_send()) lustre-OST0000-osc-MDT0000: can't precreate: rc = -5 [ 5724.994770] LustreError: 172123:0:(osp_precreate.c:1359:osp_precreate_thread()) lustre-OST0000-osc-MDT0000: cannot precreate objects: rc = -5 [ 5725.002149] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5725.712807] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5726.534169] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5727.313567] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5728.131166] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5728.978281] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5729.813909] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5730.582924] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5731.347582] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5732.089837] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5732.885763] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5733.727302] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5734.603988] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5735.312244] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5736.189767] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5737.124039] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5738.112894] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5738.881970] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5739.623818] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5740.320119] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5741.133952] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5741.811554] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5742.688401] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5743.467980] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5744.278227] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5745.123851] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5746.050416] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5746.850080] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5747.804058] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5748.608875] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5749.654410] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5750.541826] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5751.481070] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5752.230553] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5753.053950] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5753.724416] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5754.508226] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5755.262970] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5756.062187] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5756.832489] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5757.812267] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5758.763608] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5759.689613] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5760.461487] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5761.304071] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5762.075842] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5762.897522] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5763.698937] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5764.457753] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5765.372705] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5766.326695] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5767.554072] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5768.580649] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5769.739797] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5770.714931] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5771.725051] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5772.753369] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5773.804204] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5774.623711] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5775.529581] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5776.570655] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5777.910122] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5779.266742] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5780.606869] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5782.062371] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5783.434059] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5784.831882] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5786.018856] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5786.976872] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5788.030556] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5788.994381] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5790.374478] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 5790.384794] Lustre: Skipped 284 previous similar messages [ 5790.406661] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5790.417798] Lustre: Skipped 284 previous similar messages [ 5790.421079] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 5790.423845] Lustre: Skipped 284 previous similar messages [ 5790.428655] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5790.438450] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 5790.469311] Lustre: Skipped 284 previous similar messages [ 5791.480476] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5792.308778] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5793.431734] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5794.374747] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5795.370189] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5796.344660] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5797.372177] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5798.297433] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5799.102301] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5799.849628] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5800.690654] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5801.544462] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5802.476632] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5803.581268] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5804.727670] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5805.803744] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5807.001729] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5807.925146] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5808.925790] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5809.949282] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5811.247363] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5812.097868] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5813.349316] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5814.593167] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5815.711064] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5816.731805] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5817.706841] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5818.589750] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5819.609538] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5820.356724] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5821.177882] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5822.068328] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5822.918415] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5823.662372] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5824.533265] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5825.231787] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5826.061193] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5826.953944] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5827.779466] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5828.570975] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5829.309482] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5830.122863] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5830.854865] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5831.658429] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5832.414455] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5833.317779] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5834.025272] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5834.672898] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5835.416204] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5836.176997] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5836.956305] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5837.666560] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5838.412613] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5839.023322] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5839.661818] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5840.214341] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5840.768522] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5841.399674] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5842.154059] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5842.833232] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5843.490074] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5844.067184] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5844.722750] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5845.286437] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5846.022206] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5846.719703] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5847.444036] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5848.182778] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5848.890089] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5849.556260] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5850.243076] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5850.856720] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5851.568472] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5852.347862] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5852.939186] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5853.702384] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5854.390302] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5855.073312] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5855.665659] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5856.374499] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5856.983178] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5857.776915] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5858.527313] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5859.425897] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5860.181096] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5860.851131] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5861.583340] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5862.297171] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5863.040289] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5863.764100] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5864.571742] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5865.517545] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5866.514436] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5867.268853] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5868.120667] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5869.048160] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5869.906542] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5870.998960] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5871.885358] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5872.656700] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5873.635921] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5874.600366] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5875.811172] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5877.139644] LustreError: 172123:0:(osp_precreate.c:655:osp_precreate_send()) lustre-OST0000-osc-MDT0000: can't precreate: rc = -5 [ 5877.161126] LustreError: 172123:0:(osp_precreate.c:1359:osp_precreate_thread()) lustre-OST0000-osc-MDT0000: cannot precreate objects: rc = -5 [ 5877.185426] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5878.367188] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5879.684489] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5880.662284] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5881.636864] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5882.512437] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5883.643958] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5884.688565] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5885.723316] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5886.768671] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5887.863465] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5889.094179] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5890.104639] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5890.978024] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5892.197428] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5893.233477] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5894.280514] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5895.223988] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5896.457775] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5897.403622] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5898.159967] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5899.195359] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5900.274614] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5901.706824] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5903.310575] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5904.392252] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5905.785166] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5907.191208] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5908.778808] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5910.236987] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5911.897876] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5913.491980] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5914.955791] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5916.524868] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5918.027625] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5919.332694] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5921.050417] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5922.157418] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5923.247773] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5924.491711] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5925.331788] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5926.040294] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5926.711824] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5927.713652] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5928.756338] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5929.865990] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5931.171024] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5932.335153] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5933.658256] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5934.809621] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5935.509522] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5936.795126] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5937.888059] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5938.757806] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5939.429250] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5940.273481] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5940.975064] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5941.940369] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5942.752775] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5943.497518] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5944.242724] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5944.881934] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5945.515171] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5946.234128] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5947.040335] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5947.776500] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5948.622598] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5949.471209] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5950.353327] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5951.322348] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5952.231920] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5953.081820] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5953.848234] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5954.578449] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5955.671847] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5956.542921] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5957.409992] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5958.339134] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5959.329221] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5960.245736] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5961.200251] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5962.302846] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5963.298824] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5964.338027] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5965.244376] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5966.430762] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5967.459536] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5968.566500] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5969.738440] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5970.712075] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5971.915745] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5972.838734] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5973.949433] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5975.126458] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5976.113203] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5977.389163] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5978.383688] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5980.208073] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5981.793447] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5983.563066] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5985.096453] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5986.479523] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5987.855618] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5989.340897] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5990.452742] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5992.193571] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5993.343773] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5994.795989] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5995.845845] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5997.047965] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5998.269368] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5999.720381] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6001.112590] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6002.201171] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6003.287880] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6004.569253] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6005.469472] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6006.346814] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6007.610506] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6008.793715] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6010.602247] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6011.771950] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6012.760284] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6013.810207] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6014.588550] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6015.465859] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6016.430236] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6017.433926] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6018.337754] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6019.367658] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6020.071233] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6020.895825] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6021.757398] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6022.574198] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6023.762379] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6024.928626] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6025.945815] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6027.043542] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6027.815088] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6028.617851] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6029.659337] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6030.787557] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6031.954578] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6032.852644] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6033.846971] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6034.868827] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6035.835153] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6036.803979] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6037.886571] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6038.933130] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6040.003616] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6040.926057] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6041.690780] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6042.557732] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6043.405430] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6044.418834] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6045.206977] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6046.042184] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6046.779636] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6047.612059] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6048.381598] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6049.015478] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6049.789637] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6050.680923] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6051.560558] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6052.460199] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6053.277374] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6054.030931] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6054.836514] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6055.693540] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6056.522319] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6057.410962] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6058.315848] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6059.056868] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6060.021494] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6061.022798] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6062.087347] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6063.077865] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6064.115970] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6065.037949] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6066.063866] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6066.990340] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6068.262611] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6069.477860] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6070.660235] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6072.166337] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6073.681801] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6075.290025] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6076.566979] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6078.225975] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6079.505878] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6080.264671] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6081.479166] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6082.749214] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6084.371078] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6086.128271] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6087.441296] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6088.731464] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6090.043666] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6091.716277] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6092.684553] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6094.014933] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6095.028939] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6096.119646] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6097.295135] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6098.638059] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6099.618138] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6100.872328] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6102.161533] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6103.428563] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6104.487690] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6105.680951] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6106.895673] LustreError: 172123:0:(osp_precreate.c:655:osp_precreate_send()) lustre-OST0000-osc-MDT0000: can't precreate: rc = -5 [ 6106.918453] LustreError: 172123:0:(osp_precreate.c:1359:osp_precreate_thread()) lustre-OST0000-osc-MDT0000: cannot precreate objects: rc = -5 [ 6106.931856] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6108.076944] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6109.219709] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6110.654491] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6111.763603] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6112.857922] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6113.836328] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6114.854540] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6116.006446] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6117.178255] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6118.604787] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6119.525620] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6120.571754] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6121.769317] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6122.528439] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6123.428661] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6124.177538] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6125.031549] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6125.913642] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6126.874684] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6127.979437] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6128.966752] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6129.975738] LustreError: 172123:0:(osp_precreate.c:655:osp_precreate_send()) lustre-OST0000-osc-MDT0000: can't precreate: rc = -5 [ 6129.984550] LustreError: 172123:0:(osp_precreate.c:1359:osp_precreate_thread()) lustre-OST0000-osc-MDT0000: cannot precreate objects: rc = -5 [ 6129.996344] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6130.901737] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6131.962134] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6132.924788] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6133.854555] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6134.860874] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6136.076247] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6137.341958] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6138.514719] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6139.335641] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6140.311822] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6141.658416] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6142.428961] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6143.400344] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6144.222770] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6145.198648] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6146.226188] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6147.184690] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6148.113365] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6149.210159] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6150.114243] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6151.271910] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6152.165372] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6153.326829] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6154.429277] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6155.659863] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6156.566434] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6158.350480] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6159.709143] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6160.937445] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6161.898250] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6163.434181] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6164.988724] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6166.886760] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6168.171456] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6169.415833] LustreError: 172123:0:(osp_precreate.c:655:osp_precreate_send()) lustre-OST0000-osc-MDT0000: can't precreate: rc = -5 [ 6169.428837] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6169.436638] LustreError: 172123:0:(osp_precreate.c:1359:osp_precreate_thread()) lustre-OST0000-osc-MDT0000: cannot precreate objects: rc = -5 [ 6170.897903] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6172.422708] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6173.669214] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6175.654633] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6177.641095] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6179.226659] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6180.836524] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6181.988358] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6183.025515] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6184.355307] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6185.515384] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6186.476624] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6187.402773] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6188.280629] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6189.107036] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6189.944715] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6190.758241] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6191.624422] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6192.712919] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6193.742977] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6194.596973] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6195.609079] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6196.643658] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6197.430707] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6198.320949] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6199.456815] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6200.283456] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6201.225567] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6202.094202] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6203.199421] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6204.120264] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6205.090352] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6205.824660] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6206.730842] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6207.542842] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6208.781515] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6209.566794] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6210.504540] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6211.445373] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6212.447779] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6213.619870] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6214.606711] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6215.603603] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6216.579344] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6217.661815] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6218.505656] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6219.687364] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6220.705791] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6221.876933] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6222.763833] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6223.754817] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6224.732107] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6225.801639] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6226.719378] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6227.839072] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6228.909302] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6230.102583] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6231.057183] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6231.821890] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6232.705611] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6233.674542] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6234.783092] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6235.762381] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6236.491226] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6237.349536] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6238.216703] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6239.133106] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6240.034784] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6241.065936] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6242.155700] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6243.393402] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6244.428325] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6245.288273] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6246.331219] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6247.430171] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6591.435430] LustreError: 173926:0:(ofd_dev.c:1780:ofd_create_hdl()) lustre-OST0000: unable to precreate: rc = -28 [ 6591.449702] LustreError: 172123:0:(osp_precreate.c:655:osp_precreate_send()) lustre-OST0000-osc-MDT0000: can't precreate: rc = -28 [ 7001.060812] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 7001.061851] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 7001.076654] Lustre: Skipped 449 previous similar messages [ 7001.083139] Lustre: Skipped 1 previous similar message [ 7006.178810] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 7006.183891] Lustre: Skipped 1 previous similar message [ 7006.799633] Lustre: server umount lustre-OST0000 complete [ 7011.300554] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 7011.312272] Lustre: Skipped 1 previous similar message [ 7016.416895] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 7016.421813] Lustre: Skipped 1 previous similar message [ 7021.541371] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 7021.553837] Lustre: Skipped 1 previous similar message [ 7024.095148] Lustre: lustre-MDT0000 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 7024.411960] Lustre: server umount lustre-MDT0000 complete [ 7026.657907] LustreError: 225939:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 7026.688220] LustreError: 225939:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 1 previous similar message [ 7028.061254] LustreError: 172083:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1787439216 with bad export cookie 714480404597794527 [ 7028.067964] LustreError: MGC192.168.202.111@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 7028.073994] LustreError: 172083:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 7028.305762] Lustre: server umount lustre-MDT0001 complete [ 7038.074183] Lustre: DEBUG MARKER: oleg211-server.virtnet: executing unload_modules_local [ 7040.460521] Key type lgssc unregistered [ 7040.732904] LNet: 227321:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 7040.740594] LNetError: 227321:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 7040.761707] LNet: Removed LNI 192.168.202.111@tcp [ 7041.516726] Key type .llcrypt unregistered [ 7041.518105] Key type ._llcrypt unregistered [ 7065.817808] Key type ._llcrypt registered [ 7065.822603] Key type .llcrypt registered [ 7066.029440] Lustre: DEBUG MARKER: oleg211-server.virtnet: executing load_modules_local [ 7067.875207] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 7068.070092] alg: No test for adler32 (adler32-zlib) [ 7069.227738] Lustre: Lustre: Build Version: 2.17.57_63_gaeb0ce0 [ 7069.759952] LNet: Added LNI 192.168.202.111@tcp [8/256/0/180] [ 7071.599230] Key type lgssc registered [ 7072.842806] Lustre: Echo OBD driver; http://www.lustre.org/ [ 7081.789854] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 7081.820624] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 7083.737804] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 7088.818095] Lustre: DEBUG MARKER: oleg211-server.virtnet: executing set_default_debug -1 all [ 7093.219083] LustreError: 228750:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 7093.240136] LustreError: 228750:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 2 previous similar messages [ 7096.279341] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 7096.544078] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 7099.993492] Lustre: DEBUG MARKER: oleg211-server.virtnet: executing set_default_debug -1 all [ 7105.761708] Lustre: DEBUG MARKER: oleg211-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 7109.489253] Lustre: DEBUG MARKER: oleg211-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 7115.290293] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 7115.682876] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 7121.033252] Lustre: DEBUG MARKER: oleg211-server.virtnet: executing set_default_debug -1 all [ 7127.994983] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:49665 to 0x280000401:49696) [ 7130.502899] Lustre: DEBUG MARKER: oleg211-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 7137.974479] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 7140.729924] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 7140.760209] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 7140.777070] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7140.796702] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 7140.799040] LustreError: 228770:0:(osp_precreate.c:1268:osp_precreate_thread()) cfs_fail_timeout id 2107 sleeping for 20000ms [ 7143.407250] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 7143.413621] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 7143.436215] Lustre: Skipped 1 previous similar message [ 7143.988533] Lustre: lustre-OST0000: Not available for connect from 192.168.202.11@tcp (stopping) [ 7144.013993] Lustre: Skipped 1 previous similar message [ 7148.534147] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 7148.541368] Lustre: Skipped 1 previous similar message [ 7148.821607] Lustre: server umount lustre-OST0000 complete [ 7160.879198] LustreError: 228770:0:(osp_precreate.c:1268:osp_precreate_thread()) cfs_fail_timeout id 2107 awake [ 7183.425316] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 7183.811588] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 7191.110774] Lustre: DEBUG MARKER: oleg211-server.virtnet: executing set_default_debug -1 all [ 7198.212029] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7198.232659] LustreError: lustre-OST0000-osc-MDT0001: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7198.240175] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 7198.273835] Lustre: Skipped 1 previous similar message [ 7199.922638] Lustre: DEBUG MARKER: oleg211-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 7201.688261] Lustre: DEBUG MARKER: osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid in FULL state after 0 sec [ 7206.847993] Lustre: DEBUG MARKER: oleg211-client.virtnet: executing wait_import_state (FULL|IDLE) osc.lustre-OST0000-osc-ffff92438bb1e000.ost_server_uuid 50 [ 7208.902333] Lustre: DEBUG MARKER: osc.lustre-OST0000-osc-ffff92438bb1e000.ost_server_uuid in IDLE state after 0 sec [ 7213.537647] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 7213.553297] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 7213.567035] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 7213.572709] Lustre: Skipped 1 previous similar message [ 7218.676657] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 7218.678804] Lustre: Skipped 1 previous similar message [ 7218.912448] Lustre: server umount lustre-OST0000 complete [ 7223.784509] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 7223.799439] Lustre: Skipped 1 previous similar message [ 7228.614808] Lustre: server umount lustre-MDT0000 complete [ 7228.900529] LustreError: 228750:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 7228.915328] LustreError: 228750:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 1 previous similar message [ 7232.504129] LustreError: 228731:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1787439420 with bad export cookie 10300164554003752085 [ 7232.535563] LustreError: MGC192.168.202.111@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 7232.536300] LustreError: 228731:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 2 previous similar messages [ 7233.483726] Lustre: server umount lustre-MDT0001 complete [ 7251.000248] Lustre: DEBUG MARKER: oleg211-server.virtnet: executing unload_modules_local [ 7255.551418] Key type lgssc unregistered [ 7255.913716] LNet: 233960:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 7255.921861] LNetError: 233960:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 7255.941540] LNet: Removed LNI 192.168.202.111@tcp [ 7257.284105] Key type .llcrypt unregistered [ 7257.285763] Key type ._llcrypt unregistered [ 7291.079578] Key type ._llcrypt registered [ 7291.082570] Key type .llcrypt registered [ 7291.217210] Lustre: DEBUG MARKER: oleg211-server.virtnet: executing load_modules_local [ 7292.419223] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 7292.464189] alg: No test for adler32 (adler32-zlib) [ 7293.816495] Lustre: Lustre: Build Version: 2.17.57_63_gaeb0ce0 [ 7294.302841] LNet: Added LNI 192.168.202.111@tcp [8/256/0/180] [ 7296.103674] Key type lgssc registered [ 7297.951638] Lustre: Echo OBD driver; http://www.lustre.org/ [ 7309.274746] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 7309.339982] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 7311.202578] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 7317.478452] Lustre: DEBUG MARKER: oleg211-server.virtnet: executing set_default_debug -1 all [ 7321.570153] LustreError: 235689:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 7321.594283] LustreError: 235689:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 2 previous similar messages [ 7326.698593] LustreError: 235708:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 7327.784907] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 7328.206961] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 7332.814873] Lustre: DEBUG MARKER: oleg211-server.virtnet: executing set_default_debug -1 all [ 7335.942951] Lustre: Modifying parameter general.debug_raw_pointers=Y in log params [ 7342.260654] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 7342.619847] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 7348.693835] Lustre: DEBUG MARKER: oleg211-server.virtnet: executing set_default_debug -1 all [ 7356.002789] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:49698 to 0x280000401:49728) [ 7357.270951] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 7357.497746] Lustre: lustre-OST0001: new disk, initializing [ 7357.503971] Lustre: srv-lustre-OST0001: No data found on store. Initialize space. [ 7357.512978] Lustre: 238001:0:(osd_compat.c:1353:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 7357.661096] Lustre: lustre-OST0001: Imperative Recovery not enabled, recovery window 60-180 [ 7363.185584] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x00000002c0000400-0x0000000300000400]:1:ost [ 7363.198695] Lustre: cli-lustre-OST0001-super: Allocated super-sequence [0x00000002c0000400-0x0000000300000400]:1:ost] [ 7363.294590] Lustre: lustre-OST0001-osc-MDT0000: update sequence from 0x100010000 to 0x2c0000401 [ 7363.491698] Lustre: DEBUG MARKER: oleg211-server.virtnet: executing set_default_debug -1 all [ 7374.605419] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 7385.545227] Lustre: Setting parameter general.lod.*.mdt_hash=crush in log params [ 7397.944702] Lustre: server umount lustre-MDT0000 complete [ 7398.899347] LustreError: lustre-MDT0000-osp-MDT0001: operation out_update to node 0@lo failed: rc = -107 [ 7398.909301] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 7398.922109] LustreError: 235690:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 7399.411744] Lustre: lustre-MDT0000-lwp-OST0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 7399.425255] Lustre: Skipped 2 previous similar messages [ 7401.140718] LustreError: 236427:0:(osp_object.c:618:osp_attr_get()) lustre-MDT0000-osp-MDT0001: osp_attr_get update error [0x200001b72:0x1:0x0]: rc = -5 [ 7401.142385] LustreError: 235671:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1787439589 with bad export cookie 15325669584729349524 [ 7401.147340] LustreError: MGC192.168.202.111@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 7401.157542] LustreError: 236427:0:(client.c:1394:ptlrpc_import_delay_req()) @@@ IMP_CLOSED req@ffff9c91e367ca80 x1874266142383488/t0(0) o1000->lustre-MDT0000-osp-MDT0001@0@lo:24/4 lens 264/4320 e 0 to 0 dl 0 ref 2 fl Rpc:QU/200/ffffffff rc 0/-1 job:'dist_txn-1.0' uid:0 gid:0 projid:4294967295 [ 7401.204215] LustreError: 235671:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 7401.438320] Lustre: server umount lustre-MDT0001 complete [ 7415.235407] Lustre: server umount lustre-OST0000 complete [ 7429.027963] Lustre: server umount lustre-OST0001 complete [ 7455.126984] Lustre: DEBUG MARKER: oleg211-server.virtnet: executing load_modules_local [ 7465.618413] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 7466.013180] LustreError: 241114:0:(ldlm_lib.c:1190:target_handle_connect()) mylustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 7466.042262] LustreError: 241114:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 4 previous similar messages [ 7466.125786] Lustre: mylustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 7470.665125] Lustre: DEBUG MARKER: oleg211-server.virtnet: executing set_default_debug -1 all [ 7471.601245] LustreError: 241115:0:(ldlm_lib.c:1190:target_handle_connect()) mylustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 7479.827428] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 7480.351181] Lustre: mylustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 7485.237252] Lustre: DEBUG MARKER: oleg211-server.virtnet: executing set_default_debug -1 all [ 7488.232245] Lustre: 242267:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify general/debug_raw_pointers=Y (mode = 0) failed: rc = -17 [ 7495.781985] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 7505.122443] Lustre: DEBUG MARKER: oleg211-server.virtnet: executing set_default_debug -1 all [ 7505.347729] LustreError: 242622:0:(ldlm_lib.c:1190:target_handle_connect()) mylustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 7505.363313] Lustre: mylustre-OST0000: new connection from mylustre-MDT0001-mdtlov (cleaning up unused objects from 0x280000400:100 to 0x280000400:129) [ 7505.385849] LustreError: 242622:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 1 previous similar message [ 7511.584839] Lustre: mylustre-OST0000: new connection from mylustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:49698 to 0x280000401:49760) [ 7516.460394] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 7516.748291] Lustre: mylustre-OST0001: Imperative Recovery not enabled, recovery window 60-180 [ 7516.759299] Lustre: Skipped 1 previous similar message [ 7522.793858] Lustre: DEBUG MARKER: oleg211-server.virtnet: executing set_default_debug -1 all [ 7529.932875] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 7533.348692] Lustre: 244137:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify general/lod.*.mdt_hash=crush (mode = 0) failed: rc = -17 [ 7546.851595] LustreError: mylustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 7546.861966] Lustre: mylustre-MDT0000-osp-MDT0001: Connection to mylustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 7546.870816] Lustre: mylustre-MDT0000: Not available for connect from 0@lo (stopping) [ 7547.372302] Lustre: mylustre-MDT0000: Not available for connect from 0@lo (stopping) [ 7547.376758] Lustre: Skipped 3 previous similar messages [ 7549.291503] Lustre: server umount mylustre-MDT0000 complete [ 7552.486902] LustreError: 241858:0:(ldlm_lib.c:1190:target_handle_connect()) mylustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 7552.516444] LustreError: 241858:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 6 previous similar messages [ 7553.215867] LustreError: 243450:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1787439741 with bad export cookie 15325669584729352373 [ 7553.220433] LustreError: MGC192.168.202.111@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 7553.228758] LustreError: 243450:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 2 previous similar messages [ 7553.602166] Lustre: server umount mylustre-MDT0001 complete [ 7567.091734] Lustre: server umount mylustre-OST0000 complete [ 7581.516844] Lustre: server umount mylustre-OST0001 complete [ 7604.026904] Lustre: DEBUG MARKER: oleg211-server.virtnet: executing load_modules_local [ 7614.359309] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 7614.897243] LustreError: 246448:0:(ldlm_lib.c:1190:target_handle_connect()) tfs-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 7615.051766] Lustre: tfs-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 7620.292715] Lustre: DEBUG MARKER: oleg211-server.virtnet: executing set_default_debug -1 all [ 7629.989572] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 7635.763223] Lustre: DEBUG MARKER: oleg211-server.virtnet: executing set_default_debug -1 all [ 7638.963237] Lustre: 247589:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify general/debug_raw_pointers=Y (mode = 0) failed: rc = -17 [ 7645.993109] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 7653.100413] Lustre: DEBUG MARKER: oleg211-server.virtnet: executing set_default_debug -1 all [ 7655.622920] Lustre: tfs-OST0000: new connection from tfs-MDT0001-mdtlov (cleaning up unused objects from 0x280000400:100 to 0x280000400:161) [ 7660.728403] Lustre: tfs-OST0000: new connection from tfs-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:49698 to 0x280000401:49792) [ 7664.006670] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 7669.806061] Lustre: tfs-OST0001: new connection from tfs-MDT0001-mdtlov (cleaning up unused objects from 0x2c0000400:35 to 0x2c0000400:65) [ 7672.718731] Lustre: DEBUG MARKER: oleg211-server.virtnet: executing set_default_debug -1 all [ 7680.305614] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 7684.248795] Lustre: 249463:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify general/lod.*.mdt_hash=crush (mode = 0) failed: rc = -17 [ 7695.333168] Lustre: tfs-MDT0000-osp-MDT0001: Connection to tfs-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 7695.345330] Lustre: tfs-MDT0000: Not available for connect from 0@lo (stopping) [ 7695.352677] Lustre: Skipped 5 previous similar messages [ 7695.371382] Lustre: Skipped 3 previous similar messages [ 7699.287893] Lustre: server umount tfs-MDT0000 complete [ 7700.468303] LustreError: 249716:0:(ldlm_lib.c:1190:target_handle_connect()) tfs-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 7700.484638] LustreError: 249716:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 9 previous similar messages [ 7702.852046] LustreError: 246424:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1787439891 with bad export cookie 15325669584729354326 [ 7702.854595] LustreError: MGC192.168.202.111@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 7702.860477] LustreError: 246424:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 7703.160607] Lustre: server umount tfs-MDT0001 complete [ 7719.494210] Lustre: server umount tfs-OST0000 complete [ 7721.319652] Lustre: 234951:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1787439893/real 1787439893] req@ffff9c91e33f9c00 x1874266142500992/t0(0) o400->tfs-MDT0001-lwp-OST0001@0@lo:12/10 lens 224/224 e 0 to 1 dl 1787439909 ref 1 fl Rpc:XNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 [ 7721.397570] Lustre: tfs-MDT0001-lwp-OST0001: Connection to tfs-MDT0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 7721.418140] Lustre: Skipped 1 previous similar message [ 7723.551421] Lustre: 234952:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1787439895/real 1787439895] req@ffff9c91c35a1f80 x1874266142501248/t0(0) o400->tfs-MDT0001-lwp-OST0001@0@lo:12/10 lens 224/224 e 0 to 1 dl 1787439911 ref 1 fl Rpc:XNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 [ 7726.049774] Lustre: 234950:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1787439898/real 1787439898] req@ffff9c91c35a2a00 x1874266142501504/t0(0) o400->tfs-MDT0001-lwp-OST0001@0@lo:12/10 lens 224/224 e 0 to 1 dl 1787439914 ref 1 fl Rpc:XNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 [ 7727.735357] Lustre: server umount tfs-OST0001 complete [ 7753.441537] Lustre: DEBUG MARKER: oleg211-server.virtnet: executing load_modules_local [ 7767.093478] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 7768.272688] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 7768.289088] Lustre: Skipped 3 previous similar messages [ 7775.890291] Lustre: DEBUG MARKER: oleg211-server.virtnet: executing set_default_debug -1 all [ 7793.924493] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 7799.279268] Lustre: DEBUG MARKER: oleg211-server.virtnet: executing set_default_debug -1 all [ 7802.242979] Lustre: 252863:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify general/debug_raw_pointers=Y (mode = 0) failed: rc = -17 [ 7811.982898] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 7815.811586] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:49698 to 0x280000401:49824) [ 7818.894188] Lustre: lustre-OST0000: new connection from lustre-MDT0001-mdtlov (cleaning up unused objects from 0x280000400:100 to 0x280000400:193) [ 7820.631324] Lustre: DEBUG MARKER: oleg211-server.virtnet: executing set_default_debug -1 all [ 7829.479962] LustreError: 253219:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 7829.515832] LustreError: 253219:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 11 previous similar messages [ 7830.046199] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 7835.643930] Lustre: lustre-OST0001: new connection from lustre-MDT0001-mdtlov (cleaning up unused objects from 0x2c0000400:35 to 0x2c0000400:97) [ 7836.991188] Lustre: DEBUG MARKER: oleg211-server.virtnet: executing set_default_debug -1 all [ 7846.303957] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 7851.629797] Lustre: 254734:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify general/lod.*.mdt_hash=crush (mode = 0) failed: rc = -17 [ 7862.077261] Lustre: DEBUG MARKER: == conf-sanity test 104a: Make sure user defined options are reflected in mount ========================================================== 19:07:28 (1787440048) [ 7881.196596] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 7881.213978] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 7881.238321] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 7884.819111] Lustre: server umount lustre-MDT0000 complete [ 7889.837744] LustreError: 254736:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1787440078 with bad export cookie 15325669584729356244 [ 7889.844159] LustreError: MGC192.168.202.111@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 7889.854080] LustreError: 254736:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 1 previous similar message [ 7891.965389] Lustre: lustre-MDT0001: Not available for connect from 0@lo (stopping) [ 7891.978955] Lustre: Skipped 4 previous similar messages [ 7900.087202] Lustre: lustre-MDT0001: Not available for connect from 0@lo (stopping) [ 7900.095603] Lustre: Skipped 3 previous similar messages [ 7904.223148] Lustre: lustre-MDT0001 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 7904.857815] Lustre: server umount lustre-MDT0001 complete [ 7925.215347] Lustre: lustre-OST0000 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 7925.354762] Lustre: server umount lustre-OST0000 complete [ 7936.610846] Lustre: server umount lustre-OST0001 complete [ 7944.300660] Lustre: DEBUG MARKER: oleg211-server.virtnet: executing set_hostid [ 7955.900605] Lustre: DEBUG MARKER: oleg211-server.virtnet: executing load_modules_local [ 7991.338108] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 7991.559771] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 7991.587178] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 7991.663397] Lustre: lustre-MDT0000: new disk, initializing [ 7991.741901] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 188-180 [ 7991.755078] Lustre: Skipped 3 previous similar messages [ 7991.777778] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 7996.864237] Lustre: DEBUG MARKER: oleg211-server.virtnet: executing set_default_debug -1 all [ 8008.762725] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 8008.848650] Lustre: 257925:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 8008.910990] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [ 8008.915437] Lustre: Skipped 1 previous similar message [ 8008.996745] Lustre: lustre-MDT0001: new disk, initializing [ 8009.199369] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [ 8009.223248] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 8015.443523] Lustre: DEBUG MARKER: oleg211-server.virtnet: executing set_default_debug -1 all [ 8027.082614] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 8027.432463] Lustre: lustre-OST0000: new disk, initializing [ 8027.436678] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 8027.442208] Lustre: 259508:0:(osd_compat.c:1353:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 8028.939122] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [ 8028.958473] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 8029.072108] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 8035.688748] Lustre: DEBUG MARKER: oleg211-server.virtnet: executing set_default_debug -1 all [ 8048.297735] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 8048.702276] Lustre: lustre-OST0001: new disk, initializing [ 8048.713577] Lustre: srv-lustre-OST0001: No data found on store. Initialize space. [ 8048.725971] Lustre: 260379:0:(osd_compat.c:1353:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 8050.036120] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x00000002c0000400-0x0000000300000400]:1:ost [ 8050.046701] Lustre: cli-lustre-OST0001-super: Allocated super-sequence [0x00000002c0000400-0x0000000300000400]:1:ost] [ 8050.200430] Lustre: lustre-OST0001-osc-MDT0000: update sequence from 0x100010000 to 0x2c0000401 [ 8056.932315] Lustre: DEBUG MARKER: oleg211-server.virtnet: executing set_default_debug -1 all [ 8069.604410] Lustre: DEBUG MARKER: == conf-sanity test 104b: Mount uses last flock argument ========================================================== 19:10:55 (1787440255) [ 8084.239025] Lustre: DEBUG MARKER: == conf-sanity test 105: check file creation for ro and rw bind mnt pt ========================================================== 19:11:10 (1787440270) [ 8100.838557] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 8100.839151] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 8100.851894] LustreError: Skipped 1 previous similar message [ 8100.861779] Lustre: Skipped 5 previous similar messages [ 8100.871721] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 8105.711735] Lustre: server umount lustre-OST0000 complete [ 8106.483090] LustreError: 259593:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-OST0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 8106.501270] LustreError: 259593:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 11 previous similar messages [ 8118.026667] Lustre: server umount lustre-MDT0000 complete [ 8122.975883] LustreError: 257917:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1787440311 with bad export cookie 15325669584729358022 [ 8122.981242] LustreError: MGC192.168.202.111@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 8122.985766] LustreError: 257917:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 2 previous similar messages [ 8129.357392] Lustre: server umount lustre-MDT0001 complete [ 8142.631489] Lustre: DEBUG MARKER: oleg211-server.virtnet: executing unload_modules_local [ 8216.357913] Lustre: server umount lustre-OST0001 complete [ 8223.847870] Lustre: DEBUG MARKER: oleg211-server.virtnet: executing set_hostid [ 8242.638253] Lustre: DEBUG MARKER: oleg211-server.virtnet: executing load_modules_local [ 8285.346916] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 8285.581595] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 8285.601336] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 8285.681853] Lustre: lustre-MDT0000: new disk, initializing [ 8285.769962] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 8285.778294] Lustre: Skipped 3 previous similar messages [ 8285.790332] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 8291.979469] Lustre: DEBUG MARKER: oleg211-server.virtnet: executing set_default_debug -1 all [ 8305.382263] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 8305.659607] Lustre: 265225:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 8305.781277] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [ 8305.789732] Lustre: Skipped 1 previous similar message [ 8305.893256] Lustre: lustre-MDT0001: new disk, initializing [ 8306.050854] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [ 8311.683809] Lustre: DEBUG MARKER: oleg211-server.virtnet: executing set_default_debug -1 all [ 8324.513939] Lustre: DEBUG MARKER: oleg211-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 8331.817854] Lustre: DEBUG MARKER: oleg211-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 8341.083563] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 8341.477540] Lustre: lustre-OST0000: new disk, initializing [ 8341.483836] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 8341.514810] Lustre: 267193:0:(osd_compat.c:1353:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 8342.680143] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [ 8342.696016] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 8342.836984] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 8349.091774] Lustre: DEBUG MARKER: oleg211-server.virtnet: executing set_default_debug -1 all [ 8360.563347] Lustre: DEBUG MARKER: oleg211-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 8368.609030] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 8368.610827] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 8368.627188] Lustre: Skipped 4 previous similar messages [ 8368.633650] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 8368.644021] Lustre: Skipped 12 previous similar messages [ 8374.342930] Lustre: server umount lustre-OST0000 complete [ 8384.981549] Lustre: server umount lustre-MDT0000 complete [ 8389.117026] LustreError: 265217:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1787440577 with bad export cookie 15325669584729359492 [ 8389.146232] LustreError: 265217:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 1 previous similar message [ 8389.148501] LustreError: MGC192.168.202.111@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 8389.597910] Lustre: server umount lustre-MDT0001 complete [ 8401.837992] Lustre: DEBUG MARKER: oleg211-server.virtnet: executing unload_modules_local [ 8405.080322] Key type lgssc unregistered [ 8405.436963] LNet: 268959:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 8405.442844] LNetError: 268959:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 8406.506797] LNet: Removed LNI 192.168.202.111@tcp [ 8407.477177] Key type .llcrypt unregistered [ 8407.479845] Key type ._llcrypt unregistered [ 8435.866556] Key type ._llcrypt registered [ 8435.868056] Key type .llcrypt registered [ 8435.988940] Lustre: DEBUG MARKER: oleg211-server.virtnet: executing load_modules_local [ 8436.723517] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 8436.743862] alg: No test for adler32 (adler32-zlib) [ 8438.118446] Lustre: Lustre: Build Version: 2.17.57_63_gaeb0ce0 [ 8438.353465] LNet: Added LNI 192.168.202.111@tcp [8/256/0/180] [ 8440.031656] Key type lgssc registered [ 8441.384109] Lustre: Echo OBD driver; http://www.lustre.org/ [ 8450.791342] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 8450.812445] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 8452.561581] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 8457.957788] Lustre: DEBUG MARKER: oleg211-server.virtnet: executing set_default_debug -1 all [ 8461.793809] LustreError: 270420:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 8461.826125] LustreError: 270420:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 2 previous similar messages [ 8466.914316] LustreError: 270419:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 8468.143078] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 8468.633724] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 8473.875376] Lustre: DEBUG MARKER: oleg211-server.virtnet: executing set_default_debug -1 all [ 8483.289849] Lustre: DEBUG MARKER: oleg211-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 8490.177533] Lustre: DEBUG MARKER: oleg211-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 8498.080347] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 8498.566122] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 8507.612459] Lustre: DEBUG MARKER: oleg211-server.virtnet: executing set_default_debug -1 all [ 8518.252985] Lustre: DEBUG MARKER: oleg211-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 8519.248769] Lustre: Setting parameter lustre-OST0000.ost.unknown_param=50 in log lustre-OST0000 [ 8522.211489] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 8522.216841] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 8527.356584] LustreError: 272240:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-OST0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 8527.411560] LustreError: 272240:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 1 previous similar message [ 8527.618409] Lustre: server umount lustre-OST0000 complete [ 8532.463353] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 8532.466623] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 8532.484879] Lustre: Skipped 1 previous similar message [ 8532.497748] Lustre: Skipped 2 previous similar messages [ 8537.573383] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 8537.583440] Lustre: Skipped 1 previous similar message [ 8542.688436] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 8546.272419] Lustre: lustre-MDT0000 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 8546.521837] Lustre: server umount lustre-MDT0000 complete [ 8547.812358] LustreError: 270420:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 8547.835628] LustreError: 270420:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 1 previous similar message [ 8551.701475] LustreError: 270399:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1787440739 with bad export cookie 1407945029599987023 [ 8551.711982] LustreError: MGC192.168.202.111@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 8551.721140] LustreError: 270399:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 3 previous similar messages [ 8551.990754] Lustre: server umount lustre-MDT0001 complete [ 8566.591158] Lustre: DEBUG MARKER: oleg211-server.virtnet: executing unload_modules_local [ 8570.454076] Key type lgssc unregistered [ 8570.889380] LNet: 274010:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 8570.895887] LNetError: 274010:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 8570.911806] LNet: Removed LNI 192.168.202.111@tcp [ 8571.937849] Key type .llcrypt unregistered [ 8571.943607] Key type ._llcrypt unregistered [ 8594.202666] Key type ._llcrypt registered [ 8594.204406] Key type .llcrypt registered [ 8594.398952] Lustre: DEBUG MARKER: oleg211-server.virtnet: executing load_modules_local [ 8596.600716] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 8596.619949] alg: No test for adler32 (adler32-zlib) [ 8598.117555] Lustre: Lustre: Build Version: 2.17.57_63_gaeb0ce0 [ 8598.704231] LNet: Added LNI 192.168.202.111@tcp [8/256/0/180] [ 8600.471200] Key type lgssc registered [ 8602.181096] Lustre: Echo OBD driver; http://www.lustre.org/ [ 8615.703351] Lustre: lustre-OST0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 8615.763530] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 8631.583596] LustreError: Failed to get MGS log lustre-sptlrpc and no local copy. [ 8636.707912] LustreError: 275209:0:(mgc_request.c:233:do_config_log_add()) MGC192.168.202.111@tcp: failed processing log, type 4: rc = -110 [ 8663.455492] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 8672.139849] Lustre: DEBUG MARKER: oleg211-server.virtnet: executing set_default_debug -1 all [ 8682.244865] Lustre: DEBUG MARKER: oleg211-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 8685.531089] Lustre: server umount lustre-OST0000 complete [ 8708.616713] Lustre: DEBUG MARKER: oleg211-server.virtnet: executing unload_modules_local [ 8712.901225] Key type lgssc unregistered [ 8713.358543] LNet: 276848:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 8713.375452] LNetError: 276848:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 8713.415313] LNet: Removed LNI 192.168.202.111@tcp [ 8714.463592] Key type .llcrypt unregistered [ 8714.467806] Key type ._llcrypt unregistered [ 8762.830162] Key type ._llcrypt registered [ 8762.834552] Key type .llcrypt registered [ 8762.966528] Lustre: DEBUG MARKER: oleg211-server.virtnet: executing load_modules_local [ 8764.373263] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 8764.391743] alg: No test for adler32 (adler32-zlib) [ 8765.700772] Lustre: Lustre: Build Version: 2.17.57_63_gaeb0ce0 [ 8765.986742] LNet: Added LNI 192.168.202.111@tcp [8/256/0/180] [ 8767.671165] Key type lgssc registered [ 8768.761308] Lustre: Echo OBD driver; http://www.lustre.org/ [ 8783.843611] loop0: detected capacity change from 0 to 204800000 [ 8784.133657] blk_update_request: operation not supported error, dev loop0, sector 399872 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [ 8784.146764] blk_update_request: operation not supported error, dev loop0, sector 208 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [ 8784.620078] blk_update_request: operation not supported error, dev loop0, sector 160232 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [ 8785.926195] loop0: detected capacity change from 0 to 204800000 [ 8786.108924] blk_update_request: operation not supported error, dev loop0, sector 399872 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [ 8786.139027] blk_update_request: operation not supported error, dev loop0, sector 208 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [ 8786.688878] blk_update_request: operation not supported error, dev loop0, sector 160232 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [ 8787.968594] loop0: detected capacity change from 0 to 204800000 [ 8788.211391] blk_update_request: operation not supported error, dev loop0, sector 399872 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [ 8788.221373] blk_update_request: operation not supported error, dev loop0, sector 8224 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [ 8788.345569] blk_update_request: operation not supported error, dev loop0, sector 58264 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [ 8789.452331] loop0: detected capacity change from 0 to 204800000 [ 8789.720733] blk_update_request: operation not supported error, dev loop0, sector 399872 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [ 8789.730553] blk_update_request: operation not supported error, dev loop0, sector 8224 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [ 8789.883141] blk_update_request: operation not supported error, dev loop0, sector 58264 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [ 8791.187387] loop0: detected capacity change from 0 to 204800000 [ 8791.271803] LDISKFS-fs (loop0): mounted filesystem with ordered data mode. Opts: (null) [ 8796.107702] loop0: detected capacity change from 0 to 204800000 [ 8796.146758] LDISKFS-fs (loop0): mounted filesystem with ordered data mode. Opts: (null) [ 8800.849577] loop0: detected capacity change from 0 to 204800000 [ 8800.909104] LDISKFS-fs (loop0): mounted filesystem with ordered data mode. Opts: (null) [ 8805.448671] loop0: detected capacity change from 0 to 204800000 [ 8805.488190] LDISKFS-fs (loop0): mounted filesystem with ordered data mode. Opts: (null) [ 8809.968428] loop0: detected capacity change from 0 to 204800000 [ 8810.079448] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 8810.136914] LDISKFS-fs (loop0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 8810.225555] Lustre: 280265:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0000: restore index 'fld' with [0x200000001:0x3:0x0]: rc = 0 [ 8810.232247] Lustre: 280265:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0000: restore index '0x2010000' with [0x200000003:0xc:0x0]: rc = 0 [ 8810.243261] Lustre: 280265:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0000: restore index '0x10000' with [0x200000003:0xa:0x0]: rc = 0 [ 8810.258227] Lustre: 280265:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0000: restore index '0x1010000-MDT0000' with [0x200000005:0x2:0x0]: rc = 0 [ 8810.274758] Lustre: 280265:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0000: restore index '0x10000-MDT0000' with [0x200000005:0x1:0x0]: rc = 0 [ 8810.284379] Lustre: 280265:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0000: restore index '0x2010000-MDT0000' with [0x200000005:0x3:0x0]: rc = 0 [ 8810.291953] Lustre: 280265:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0000: restore index '0x1010000' with [0x200000003:0xb:0x0]: rc = 0 [ 8810.298855] Lustre: 280265:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0000: restore index 'nodemap' with [0x200000003:0x3:0x0]: rc = 0 [ 8810.305305] Lustre: 280265:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0000: restore index '0x20000' with [0x200000006:0x20000:0x0]: rc = 0 [ 8810.313561] Lustre: 280265:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0000: restore index '0x2020000' with [0x200000006:0x2020000:0x0]: rc = 0 [ 8810.321968] Lustre: 280265:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0000: restore index '0x1020000' with [0x200000006:0x1020000:0x0]: rc = 0 [ 8810.330346] Lustre: 280265:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0000: restore index '0x2010000' with [0x200000006:0x2010000:0x0]: rc = 0 [ 8810.344725] Lustre: 280265:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0000: restore index '0x10000' with [0x200000006:0x10000:0x0]: rc = 0 [ 8810.352488] Lustre: 280265:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0000: restore index '0x1010000' with [0x200000006:0x1010000:0x0]: rc = 0 [ 8817.155107] Lustre: server umount MGS complete [ 8818.213168] loop0: detected capacity change from 0 to 204800000 [ 8818.365781] LDISKFS-fs (loop0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 8819.023882] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 8819.089558] Lustre: lustre-MDT0000: Aborting client recovery [ 8820.317248] loop1: detected capacity change from 0 to 204800000 [ 8820.779896] LDISKFS-fs (loop1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 8820.954942] Lustre: 280698:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0001: restore index 'fld' with [0x200000001:0x3:0x0]: rc = 0 [ 8820.965392] Lustre: 280698:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0001: restore index '0x10000' with [0x200000003:0x4:0x0]: rc = 0 [ 8820.980428] Lustre: 280698:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0001: restore index '0x2010000-MDT0001' with [0x200000005:0x3:0x0]: rc = 0 [ 8820.993974] Lustre: 280698:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0001: restore index '0x2010000' with [0x200000003:0x6:0x0]: rc = 0 [ 8821.014676] Lustre: 280698:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0001: restore index '0x1010000' with [0x200000003:0x5:0x0]: rc = 0 [ 8821.027669] Lustre: 280698:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0001: restore index '0x1010000-MDT0001' with [0x200000005:0x2:0x0]: rc = 0 [ 8821.037947] Lustre: 280698:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0001: restore index '0x10000-MDT0001' with [0x200000005:0x1:0x0]: rc = 0 [ 8821.065325] Lustre: Found index 1 for lustre-MDT0001, updating log [ 8821.359604] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 8821.422306] Lustre: lustre-MDT0001: Aborting client recovery [ 8822.172456] loop2: detected capacity change from 0 to 204800000 [ 8822.444905] LDISKFS-fs (loop2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 8822.577677] Lustre: 280777:0:(osd_scrub.c:1854:osd_index_restore()) lustre-OST0000: restore index 'fld' with [0x200000001:0x3:0x0]: rc = 0 [ 8822.587912] Lustre: 280777:0:(osd_scrub.c:1854:osd_index_restore()) lustre-OST0000: restore index '0x20000' with [0x200000003:0x5:0x0]: rc = 0 [ 8822.601554] Lustre: 280777:0:(osd_scrub.c:1854:osd_index_restore()) lustre-OST0000: restore index '0x20000-OST0000' with [0x200000005:0x1:0x0]: rc = 0 [ 8822.611577] Lustre: 280777:0:(osd_scrub.c:1854:osd_index_restore()) lustre-OST0000: restore index '0x2020000-OST0000' with [0x200000005:0x3:0x0]: rc = 0 [ 8822.620919] Lustre: 280777:0:(osd_scrub.c:1854:osd_index_restore()) lustre-OST0000: restore index '0x2020000' with [0x200000003:0x7:0x0]: rc = 0 [ 8822.642963] Lustre: 280777:0:(osd_scrub.c:1854:osd_index_restore()) lustre-OST0000: restore index '0x1020000' with [0x200000003:0x6:0x0]: rc = 0 [ 8822.658610] Lustre: 280777:0:(osd_scrub.c:1854:osd_index_restore()) lustre-OST0000: restore index '0x1020000-OST0000' with [0x200000005:0x2:0x0]: rc = 0 [ 8822.874961] Lustre: Found index 0 for lustre-OST0000, updating log [ 8822.961162] Lustre: 280795:0:(osd_compat.c:1353:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 8823.064339] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 8823.131471] Lustre: lustre-OST0000: Aborting recovery [ 8824.557958] loop3: detected capacity change from 0 to 204800000 [ 8824.802690] LDISKFS-fs (loop3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 8824.880748] Lustre: 280860:0:(osd_scrub.c:1854:osd_index_restore()) lustre-OST0001: restore index 'fld' with [0x200000001:0x3:0x0]: rc = 0 [ 8824.907669] Lustre: 280860:0:(osd_scrub.c:1854:osd_index_restore()) lustre-OST0001: restore index '0x2020000-OST0001' with [0x200000005:0x3:0x0]: rc = 0 [ 8824.935408] Lustre: 280860:0:(osd_scrub.c:1854:osd_index_restore()) lustre-OST0001: restore index '0x20000' with [0x200000003:0x5:0x0]: rc = 0 [ 8824.942664] Lustre: 280860:0:(osd_scrub.c:1854:osd_index_restore()) lustre-OST0001: restore index '0x1020000' with [0x200000003:0x6:0x0]: rc = 0 [ 8824.961609] Lustre: 280860:0:(osd_scrub.c:1854:osd_index_restore()) lustre-OST0001: restore index '0x2020000' with [0x200000003:0x7:0x0]: rc = 0 [ 8824.994576] Lustre: 280860:0:(osd_scrub.c:1854:osd_index_restore()) lustre-OST0001: restore index '0x1020000-OST0001' with [0x200000005:0x2:0x0]: rc = 0 [ 8825.028499] Lustre: 280860:0:(osd_scrub.c:1854:osd_index_restore()) lustre-OST0001: restore index '0x20000-OST0001' with [0x200000005:0x1:0x0]: rc = 0 [ 8825.118313] Lustre: Found index 1 for lustre-OST0001, updating log [ 8825.213264] Lustre: 280871:0:(osd_compat.c:1353:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 8825.356552] Lustre: lustre-OST0001: Imperative Recovery not enabled, recovery window 60-180 [ 8825.387788] Lustre: lustre-OST0001: Aborting recovery [ 8825.397374] LustreError: 280642:0:(lod_dev.c:511:lod_sub_recovery_thread()) lustre-MDT0001-osp-MDT0000: get update log duration 6, retries 0, failed: rc = -108 [ 8829.424693] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x0:6 to 0x0:33) [ 8829.434793] Lustre: lustre-OST0001: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x0:6 to 0x0:33) [ 8829.493942] Lustre: lustre-OST0001-osc-MDT0000: update sequence from 0x100010000 to 0x2c0000bd0 [ 8829.506434] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000bd0 [ 8831.472500] Lustre: lustre-OST0001: new connection from lustre-MDT0001-mdtlov (cleaning up unused objects from 0x2c0000400:5 to 0x2c0000400:33) [ 8831.474179] Lustre: lustre-OST0000: new connection from lustre-MDT0001-mdtlov (cleaning up unused objects from 0x280000400:5 to 0x280000400:33) [ 8835.281955] LustreError: 280609:0:(mdt_identity.c:115:mdt_identity_do_upcall()) lustre-MDT0000: error invoking upcall /root/Work/Lustre/R5/lustre-release/lustre/tests/../utils/l_getidentity lustre-MDT0000 0: rc -2; check /proc/fs/lustre/mdt/lustre-MDT0000/identity_upcall, time 411us: rc = -2 [ 8835.785098] LustreError: 280608:0:(mdt_identity.c:115:mdt_identity_do_upcall()) lustre-MDT0000: error invoking upcall /root/Work/Lustre/R5/lustre-release/lustre/tests/../utils/l_getidentity lustre-MDT0000 0: rc -2; check /proc/fs/lustre/mdt/lustre-MDT0000/identity_upcall, time 1961us: rc = -2 [ 8835.808528] LustreError: 280608:0:(mdt_identity.c:115:mdt_identity_do_upcall()) Skipped 15 previous similar messages [ 8851.936756] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 8851.939195] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 8851.939366] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 8851.965654] Lustre: Skipped 2 previous similar messages [ 8854.967318] Lustre: server umount lustre-MDT0000 complete [ 8855.953672] LustreError: 280593:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1787441044 with bad export cookie 15218243290147724688 [ 8855.974993] LustreError: MGC192.168.202.111@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 8857.055678] LustreError: 280608:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 8857.058602] Lustre: lustre-MDT0001-lwp-OST0000: Connection to lustre-MDT0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 8857.063065] Lustre: lustre-MDT0001: Not available for connect from 0@lo (stopping) [ 8857.063075] Lustre: Skipped 3 previous similar messages [ 8857.085636] LustreError: 280608:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 3 previous similar messages [ 8857.098245] Lustre: Skipped 2 previous similar messages [ 8860.063935] LustreError: 280606:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 8860.064488] Lustre: lustre-MDT0001: Not available for connect from 0@lo (stopping) [ 8860.088062] Lustre: Skipped 2 previous similar messages [ 8865.257448] LustreError: 280606:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 8865.259283] Lustre: lustre-MDT0001: Not available for connect from 0@lo (stopping) [ 8865.279963] LustreError: 280606:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 2 previous similar messages [ 8865.301187] Lustre: Skipped 1 previous similar message [ 8870.367185] Lustre: lustre-MDT0001 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 8870.800056] Lustre: server umount lustre-MDT0001 complete [ 8886.239504] Lustre: lustre-OST0000 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 8886.477829] Lustre: server umount lustre-OST0000 complete [ 8894.146627] Lustre: server umount lustre-OST0001 complete [ 8903.258924] Lustre: DEBUG MARKER: == conf-sanity test 109a: test lctl clear_conf fsname ==== 19:24:49 (1787441089) [ 8938.251802] Lustre: DEBUG MARKER: oleg211-server.virtnet: executing set_hostid [ 8951.628943] Lustre: DEBUG MARKER: oleg211-server.virtnet: executing load_modules_local [ 8989.645213] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 8989.964546] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 8990.025735] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 8990.112891] Lustre: lustre-MDT0000: new disk, initializing [ 8990.238560] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 8990.272105] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 8995.863853] Lustre: DEBUG MARKER: oleg211-server.virtnet: executing set_default_debug -1 all [ 9007.831565] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 9008.002623] Lustre: 284946:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 9008.104436] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [ 9008.107854] Lustre: Skipped 1 previous similar message [ 9008.211353] Lustre: lustre-MDT0001: new disk, initializing [ 9008.305765] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 9008.352860] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [ 9008.363105] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 9013.150603] Lustre: DEBUG MARKER: oleg211-server.virtnet: executing set_default_debug -1 all [ 9023.684517] Lustre: DEBUG MARKER: oleg211-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 9029.969580] Lustre: DEBUG MARKER: oleg211-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 9038.743571] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 9039.230629] Lustre: lustre-OST0000: new disk, initializing [ 9039.237486] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 9039.242941] Lustre: 286911:0:(osd_compat.c:1353:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 9039.368625] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 9046.107576] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [ 9046.149841] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 9046.373585] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 9047.393691] Lustre: DEBUG MARKER: oleg211-server.virtnet: executing set_default_debug -1 all [ 9059.247455] Lustre: DEBUG MARKER: oleg211-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 9065.695262] Lustre: Setting parameter lustre-MDT0000.mdd.atime_diff=62 in log lustre-MDT0000 [ 9079.777793] Lustre: Modifying parameter lustre-MDT0000.mdd.atime_diff=63 in log lustre-MDT0000 [ 9086.159300] Lustre: Setting parameter lustre-client.llite.max_read_ahead_mb=32 in log lustre-client [ 9138.658787] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 9138.661203] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 9138.681949] Lustre: Skipped 1 previous similar message [ 9138.695202] Lustre: Skipped 3 previous similar messages [ 9144.727121] Lustre: server umount lustre-OST0000 complete [ 9148.896353] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 9148.897971] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 9148.914707] Lustre: Skipped 1 previous similar message [ 9148.940607] Lustre: Skipped 3 previous similar messages [ 9163.231149] Lustre: lustre-MDT0000 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 9163.646142] Lustre: server umount lustre-MDT0000 complete [ 9164.257536] LustreError: 284956:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 9164.275769] LustreError: 284956:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 3 previous similar messages [ 9168.250287] LustreError: 284937:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1787441356 with bad export cookie 15218243290147727467 [ 9168.253724] LustreError: MGC192.168.202.111@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 9168.259872] LustreError: 284937:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 8 previous similar messages [ 9168.863224] Lustre: server umount lustre-MDT0001 complete [ 9178.225942] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 9183.246376] Lustre: DEBUG MARKER: oleg211-server.virtnet: executing set_default_debug -1 all [ 9189.254797] Lustre: server umount MGS complete [ 9201.488527] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 9201.966790] LustreError: 290783:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 9202.025746] LustreError: 290796:0:(qmt_pool.c:1639:qmt_pool_add_rem()) lustre-QMT0000: can't remove lustre-OST0000_UUID pool 'pool1': rc = -22 [ 9202.138567] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 9207.119211] Lustre: DEBUG MARKER: oleg211-server.virtnet: executing set_default_debug -1 all [ 9212.392377] LustreError: 290783:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 9212.414813] LustreError: 290783:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 2 previous similar messages [ 9216.332455] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 9221.915341] Lustre: DEBUG MARKER: oleg211-server.virtnet: executing set_default_debug -1 all [ 9230.198509] Lustre: DEBUG MARKER: oleg211-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 9236.647243] Lustre: DEBUG MARKER: oleg211-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 9244.760600] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 9252.476151] Lustre: DEBUG MARKER: oleg211-server.virtnet: executing set_default_debug -1 all [ 9261.958978] Lustre: DEBUG MARKER: oleg211-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 9290.733679] Lustre: server umount lustre-OST0000 complete [ 9297.386840] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 9297.397378] Lustre: Skipped 1 previous similar message [ 9297.400718] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 9297.415675] Lustre: Skipped 5 previous similar messages [ 9304.647585] Lustre: server umount lustre-MDT0000 complete [ 9307.616486] LustreError: 290780:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 9307.628111] LustreError: 290780:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 1 previous similar message [ 9307.831304] LustreError: 293620:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1787441496 with bad export cookie 15218243290147729308 [ 9307.838177] LustreError: MGC192.168.202.111@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 9307.843106] LustreError: 293620:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 9318.500976] Lustre: server umount lustre-MDT0001 complete [ 9333.406793] Lustre: DEBUG MARKER: oleg211-server.virtnet: executing unload_modules_local [ 9337.336240] Key type lgssc unregistered [ 9337.649640] LNet: 294731:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 9337.653518] LNetError: 294731:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 9337.686055] LNet: Removed LNI 192.168.202.111@tcp [ 9338.762300] Key type .llcrypt unregistered [ 9338.764806] Key type ._llcrypt unregistered [ 9387.759463] Key type ._llcrypt registered [ 9387.762227] Key type .llcrypt registered [ 9387.873870] Lustre: DEBUG MARKER: oleg211-server.virtnet: executing set_hostid [ 9403.154723] Lustre: DEBUG MARKER: oleg211-server.virtnet: executing load_modules_local [ 9404.021929] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 9404.053072] alg: No test for adler32 (adler32-zlib) [ 9405.112732] Lustre: Lustre: Build Version: 2.17.57_63_gaeb0ce0 [ 9405.450768] LNet: Added LNI 192.168.202.111@tcp [8/256/0/180] [ 9407.183287] Key type lgssc registered [ 9408.674376] Lustre: Echo OBD driver; http://www.lustre.org/ [ 9445.771067] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 9445.800326] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 9447.069257] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 9447.090227] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 9447.141916] Lustre: lustre-MDT0000: new disk, initializing [ 9447.202837] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 9447.215104] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 9451.992545] Lustre: DEBUG MARKER: oleg211-server.virtnet: executing set_default_debug -1 all [ 9464.095197] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 9464.285848] Lustre: 298694:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 9464.404137] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [ 9464.407063] Lustre: Skipped 1 previous similar message [ 9464.492592] Lustre: lustre-MDT0001: new disk, initializing [ 9464.559349] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 9464.587175] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [ 9464.602860] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 9469.254666] Lustre: DEBUG MARKER: oleg211-server.virtnet: executing set_default_debug -1 all [ 9478.597400] Lustre: DEBUG MARKER: oleg211-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 9484.459355] Lustre: DEBUG MARKER: oleg211-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 9490.645398] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 9490.904477] Lustre: lustre-OST0000: new disk, initializing [ 9490.907917] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 9490.921567] Lustre: 300661:0:(osd_compat.c:1353:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 9490.997764] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 9496.394731] Lustre: DEBUG MARKER: oleg211-server.virtnet: executing set_default_debug -1 all [ 9501.247203] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [ 9501.253479] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 9501.315214] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 9505.360342] Lustre: DEBUG MARKER: oleg211-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 9513.149231] Lustre: Setting parameter lustre-MDT0000.mdd.atime_diff=62 in log lustre-MDT0000 [ 9524.984957] Lustre: Modifying parameter lustre-MDT0000.mdd.atime_diff=63 in log lustre-MDT0000 [ 9530.291961] Lustre: Setting parameter lustre-client.llite.max_read_ahead_mb=32 in log lustre-client [ 9536.685273] Lustre: Modifying parameter lustre-client.llite.max_read_ahead_mb=64 in log lustre-client [ 9588.203358] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 9588.212406] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 9588.212475] Lustre: Skipped 1 previous similar message [ 9588.220958] Lustre: Skipped 1 previous similar message [ 9590.344645] Lustre: server umount lustre-OST0000 complete [ 9597.922233] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 9597.938749] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 9597.981272] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 9600.269395] Lustre: server umount lustre-MDT0000 complete [ 9604.638372] LustreError: 298685:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1787441792 with bad export cookie 1454601619201029385 [ 9604.640543] LustreError: MGC192.168.202.111@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 9604.660933] LustreError: 298685:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 2 previous similar messages [ 9604.998223] Lustre: server umount lustre-MDT0001 complete [ 9615.028875] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 9619.929151] Lustre: DEBUG MARKER: oleg211-server.virtnet: executing set_default_debug -1 all [ 9624.299678] Lustre: server umount MGS complete [ 9633.961718] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 9634.441903] LustreError: 304431:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 9634.462066] LustreError: 304431:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 3 previous similar messages [ 9634.485190] LustreError: 304444:0:(qmt_pool.c:1639:qmt_pool_add_rem()) lustre-QMT0000: can't remove lustre-OST0000_UUID pool 'pool1': rc = -22 [ 9634.555214] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 9639.526233] Lustre: DEBUG MARKER: oleg211-server.virtnet: executing set_default_debug -1 all [ 9639.904651] LustreError: 304432:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 9644.007972] LustreError: 304431:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 9647.124351] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 9647.455178] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 9652.009270] Lustre: DEBUG MARKER: oleg211-server.virtnet: executing set_default_debug -1 all [ 9659.005537] Lustre: DEBUG MARKER: oleg211-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 9664.825231] Lustre: DEBUG MARKER: oleg211-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 9672.200811] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 9672.615579] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 9679.486110] Lustre: DEBUG MARKER: oleg211-server.virtnet: executing set_default_debug -1 all [ 9688.072178] Lustre: DEBUG MARKER: oleg211-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 9720.800959] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 9720.824119] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 9720.848578] Lustre: Skipped 1 previous similar message [ 9720.856612] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 9720.867360] Lustre: Skipped 2 previous similar messages [ 9722.856826] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 9722.858385] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 9722.858827] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 9722.888346] Lustre: Skipped 1 previous similar message [ 9725.669864] Lustre: server umount lustre-OST0000 complete [ 9729.504957] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 9729.514878] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 9729.534939] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 9738.212072] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 9738.220576] Lustre: Skipped 3 previous similar messages [ 9743.328316] Lustre: lustre-MDT0000 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 9743.593439] Lustre: server umount lustre-MDT0000 complete [ 9747.764411] LustreError: 304412:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1787441935 with bad export cookie 1454601619201031219 [ 9747.774205] LustreError: MGC192.168.202.111@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 9747.777376] LustreError: 304412:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 9748.194839] Lustre: server umount lustre-MDT0001 complete [ 9759.128476] Lustre: DEBUG MARKER: oleg211-server.virtnet: executing unload_modules_local [ 9762.413361] Key type lgssc unregistered [ 9762.731540] LNet: 308385:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 9762.747248] LNetError: 308385:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 9763.818357] LNet: Removed LNI 192.168.202.111@tcp [ 9764.746368] Key type .llcrypt unregistered [ 9764.749024] Key type ._llcrypt unregistered [ 9787.690513] Key type ._llcrypt registered [ 9787.698782] Key type .llcrypt registered [ 9787.854078] Lustre: DEBUG MARKER: oleg211-server.virtnet: executing set_hostid [ 9802.342772] Lustre: DEBUG MARKER: oleg211-server.virtnet: executing load_modules_local [ 9803.401775] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 9803.484905] alg: No test for adler32 (adler32-zlib) [ 9804.635587] Lustre: Lustre: Build Version: 2.17.57_63_gaeb0ce0 [ 9804.896827] LNet: Added LNI 192.168.202.111@tcp [8/256/0/180] [ 9806.567234] Key type lgssc registered [ 9808.186121] Lustre: Echo OBD driver; http://www.lustre.org/ [ 9833.961701] Lustre: DEBUG MARKER: oleg211-server.virtnet: executing load_modules_local [ 9851.109442] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 9851.132115] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 9852.500262] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 9852.545280] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 9852.691296] Lustre: lustre-MDT0000: new disk, initializing [ 9852.757229] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 9852.785678] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 9857.393916] Lustre: DEBUG MARKER: oleg211-server.virtnet: executing set_default_debug -1 all [ 9870.439483] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 9870.761842] Lustre: lustre-OST0000: new disk, initializing [ 9870.766945] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 9870.770467] Lustre: Skipped 1 previous similar message [ 9870.781489] Lustre: 312440:0:(osd_compat.c:1353:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 9870.866216] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 9876.035621] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:0:ost [ 9876.047032] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:0:ost] [ 9876.091746] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x240000400 [ 9878.238585] Lustre: DEBUG MARKER: oleg211-server.virtnet: executing set_default_debug -1 all [10700.270428] Lustre: lustre-MDT0000-lwp-OST0000: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [10700.293937] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [10702.195742] Lustre: server umount lustre-MDT0000 complete [10706.571254] LustreError: 311624:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1787442894 with bad export cookie 12505057233047733741 [10706.576583] LustreError: MGC192.168.202.111@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [10706.580124] LustreError: 311624:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 2 previous similar messages [10706.747394] Lustre: server umount lustre-OST0000 complete [10734.399227] Lustre: DEBUG MARKER: oleg211-server.virtnet: executing set_hostid [10747.206457] Lustre: DEBUG MARKER: oleg211-server.virtnet: executing load_modules_local [10785.679912] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [10786.105717] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [10786.137814] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [10786.220372] Lustre: lustre-MDT0000: new disk, initializing [10786.314146] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [10786.329575] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [10790.695807] Lustre: DEBUG MARKER: oleg211-server.virtnet: executing set_default_debug -1 all [10802.907312] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [10803.067535] Lustre: 316769:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [10803.121326] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [10803.128884] Lustre: Skipped 1 previous similar message [10803.200833] Lustre: lustre-MDT0001: new disk, initializing [10803.285793] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [10803.324641] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [10803.337207] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [10807.224814] Lustre: DEBUG MARKER: oleg211-server.virtnet: executing set_default_debug -1 all [10816.782477] Lustre: DEBUG MARKER: oleg211-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [10823.567444] Lustre: DEBUG MARKER: oleg211-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [10830.564862] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [10830.909805] Lustre: lustre-OST0000: new disk, initializing [10830.917196] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [10830.926234] Lustre: 318738:0:(osd_compat.c:1353:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [10831.038252] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [10838.745970] Lustre: DEBUG MARKER: oleg211-server.virtnet: executing set_default_debug -1 all [10840.618200] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [10840.624205] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [10840.729178] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [10848.387850] Lustre: DEBUG MARKER: oleg211-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [10852.880342] Lustre: DEBUG MARKER: oleg211-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 [10853.094776] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [10858.832450] Lustre: DEBUG MARKER: oleg211-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid 50 [10859.025831] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [10861.032648] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [10861.050311] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [10861.064815] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [10861.546296] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [10866.661353] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [10866.678423] Lustre: Skipped 2 previous similar messages [10871.794193] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [10871.810900] Lustre: Skipped 1 previous similar message [10875.363466] Lustre: lustre-OST0000 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [10875.648470] Lustre: server umount lustre-OST0000 complete [10882.016869] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [10882.036717] Lustre: Skipped 1 previous similar message [10882.049828] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [10882.059552] Lustre: Skipped 1 previous similar message [10886.916378] Lustre: server umount lustre-MDT0000 complete [10887.141773] LustreError: 316780:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [10887.170767] LustreError: 316780:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 3 previous similar messages [10891.934296] LustreError: 317563:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1787443080 with bad export cookie 12505057233050674560 [10891.946455] LustreError: MGC192.168.202.111@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [10891.952476] LustreError: 317563:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [10892.439090] Lustre: server umount lustre-MDT0001 complete [10897.170981] Lustre: DEBUG MARKER: SKIP: conf-sanity test_111 skipping SLOW test 111 [10899.465917] Lustre: DEBUG MARKER: == conf-sanity test 112a: mount OST with no_create option ========================================================== 19:58:05 (1787443085) [10907.013591] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [10907.470762] LustreError: 321051:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [10907.539336] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [10912.058726] Lustre: DEBUG MARKER: oleg211-server.virtnet: executing set_default_debug -1 all [10912.736943] LustreError: 321052:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [10916.849897] LustreError: 321051:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [10920.598279] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [10921.116204] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [10926.620540] Lustre: DEBUG MARKER: oleg211-server.virtnet: executing set_default_debug -1 all [10934.298217] Lustre: DEBUG MARKER: oleg211-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [10939.558951] Lustre: DEBUG MARKER: oleg211-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [10946.036450] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [10946.355241] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [10951.864809] Lustre: DEBUG MARKER: oleg211-server.virtnet: executing set_default_debug -1 all [10958.649903] Lustre: DEBUG MARKER: oleg211-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [10966.072515] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [10966.235310] Lustre: lustre-OST0001: new disk, initializing [10966.239189] Lustre: srv-lustre-OST0001: No data found on store. Initialize space. [10966.245458] Lustre: 323887:0:(osd_compat.c:1353:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [10966.345138] Lustre: lustre-OST0001: Imperative Recovery not enabled, recovery window 60-180 [10971.987038] Lustre: DEBUG MARKER: oleg211-server.virtnet: executing set_default_debug -1 all [10975.902066] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x00000002c0000400-0x0000000300000400]:1:ost [10975.911027] Lustre: cli-lustre-OST0001-super: Allocated super-sequence [0x00000002c0000400-0x0000000300000400]:1:ost] [10975.966954] LustreError: 324405:0:(osp_precreate.c:974:osp_precreate_cleanup_orphans()) lustre-OST0001-osc-MDT0001: cannot cleanup orphans: rc = -1 [10977.032740] LustreError: 324405:0:(osp_precreate.c:974:osp_precreate_cleanup_orphans()) lustre-OST0001-osc-MDT0001: cannot cleanup orphans: rc = -1 [10977.051822] LustreError: 324405:0:(osp_precreate.c:974:osp_precreate_cleanup_orphans()) Skipped 1 previous similar message [10978.081558] LustreError: 324405:0:(osp_precreate.c:974:osp_precreate_cleanup_orphans()) lustre-OST0001-osc-MDT0001: cannot cleanup orphans: rc = -1 [10978.097948] LustreError: 324405:0:(osp_precreate.c:974:osp_precreate_cleanup_orphans()) Skipped 2 previous similar messages [10980.133356] LustreError: 324413:0:(osp_precreate.c:974:osp_precreate_cleanup_orphans()) lustre-OST0001-osc-MDT0000: cannot cleanup orphans: rc = -1 [10980.155221] LustreError: 324413:0:(osp_precreate.c:974:osp_precreate_cleanup_orphans()) Skipped 2 previous similar messages [10982.766904] Lustre: DEBUG MARKER: oleg211-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 [10983.059991] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [10984.297858] LustreError: 324413:0:(osp_precreate.c:974:osp_precreate_cleanup_orphans()) lustre-OST0001-osc-MDT0000: cannot cleanup orphans: rc = -1 [10984.312200] LustreError: 324413:0:(osp_precreate.c:974:osp_precreate_cleanup_orphans()) Skipped 8 previous similar messages [10988.843821] Lustre: DEBUG MARKER: oleg211-client.virtnet: executing wait_import_state (FULL|IDLE) osc.lustre-OST0000-osc-ffff924390a14000.ost_server_uuid 50 [10990.554531] Lustre: DEBUG MARKER: osc.lustre-OST0000-osc-ffff924390a14000.ost_server_uuid in FULL state after 0 sec [10992.486776] LustreError: 324405:0:(osp_precreate.c:974:osp_precreate_cleanup_orphans()) lustre-OST0001-osc-MDT0001: cannot cleanup orphans: rc = -1 [10992.505577] LustreError: 324405:0:(osp_precreate.c:974:osp_precreate_cleanup_orphans()) Skipped 14 previous similar messages [10996.042839] Lustre: DEBUG MARKER: oleg211-client.virtnet: executing wait_import_state (FULL|IDLE) osc.lustre-OST0001-osc-ffff924390a14000.ost_server_uuid 50 [10997.608537] Lustre: DEBUG MARKER: osc.lustre-OST0001-osc-ffff924390a14000.ost_server_uuid in FULL state after 0 sec [11004.913395] Lustre: lustre-OST0001-osc-MDT0000: update sequence from 0x100010000 to 0x2c0000401 [11015.137041] LustreError: lustre-OST0001-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [11015.146803] Lustre: lustre-OST0001-osc-MDT0001: Connection to lustre-OST0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [11015.156956] Lustre: Skipped 1 previous similar message [11015.162526] Lustre: lustre-OST0001: Not available for connect from 0@lo (stopping) [11019.897511] Lustre: server umount lustre-OST0001 complete [11022.312961] LustreError: 322874:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [11022.332913] LustreError: 322874:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 1 previous similar message [11027.439652] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [11027.441990] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [11027.454954] Lustre: Skipped 2 previous similar messages [11027.468933] LustreError: Skipped 1 previous similar message [11032.562918] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [11032.569333] LustreError: 322871:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [11032.577188] Lustre: Skipped 7 previous similar messages [11032.611513] LustreError: 322871:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 4 previous similar messages [11033.204964] Lustre: server umount lustre-OST0000 complete [11037.670560] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [11037.686615] Lustre: Skipped 1 previous similar message [11043.685490] Lustre: server umount lustre-MDT0000 complete [11047.816733] LustreError: 321033:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1787443236 with bad export cookie 12505057233050675435 [11047.830118] LustreError: MGC192.168.202.111@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [11047.831187] LustreError: 321033:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 1 previous similar message [11048.173829] Lustre: server umount lustre-MDT0001 complete [11059.204854] Lustre: DEBUG MARKER: oleg211-server.virtnet: executing unload_modules_local [11062.528964] Key type lgssc unregistered [11062.824973] LNet: 326579:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [11062.842425] LNetError: 326579:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [11062.863988] LNet: Removed LNI 192.168.202.111@tcp [11063.900722] Key type .llcrypt unregistered [11063.903931] Key type ._llcrypt unregistered [11090.574179] Key type ._llcrypt registered [11090.578912] Key type .llcrypt registered [11090.786361] Lustre: DEBUG MARKER: oleg211-server.virtnet: executing load_modules_local [11092.359350] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [11092.374797] alg: No test for adler32 (adler32-zlib) [11093.507744] Lustre: Lustre: Build Version: 2.17.57_63_gaeb0ce0 [11093.821415] LNet: Added LNI 192.168.202.111@tcp [8/256/0/180] [11095.655199] Key type lgssc registered [11096.646615] Lustre: Echo OBD driver; http://www.lustre.org/ [11108.835662] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [11108.885461] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [11110.732618] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [11116.449690] Lustre: DEBUG MARKER: oleg211-server.virtnet: executing set_default_debug -1 all [11120.099822] LustreError: 328015:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [11120.127520] LustreError: 328015:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 2 previous similar messages [11125.221085] LustreError: 328014:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [11127.097274] Lustre: DEBUG MARKER: oleg211-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [11130.337776] LustreError: 328015:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [11134.816283] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [11135.243532] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [11140.435702] Lustre: DEBUG MARKER: oleg211-server.virtnet: executing set_default_debug -1 all [11150.401695] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [11150.977214] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [11156.085845] LustreError: 329696:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [11156.132331] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:36 to 0x280000401:65) [11158.418339] Lustre: DEBUG MARKER: oleg211-server.virtnet: executing set_default_debug -1 all [11160.229110] LustreError: 329696:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [11160.266378] LustreError: 329696:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 1 previous similar message [11165.700210] Lustre: DEBUG MARKER: oleg211-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [11170.793872] LustreError: 329697:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [11170.823028] LustreError: 329697:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 3 previous similar messages [11173.242782] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [11173.475102] Lustre: lustre-OST0001: Imperative Recovery not enabled, recovery window 60-180 [11178.521306] Lustre: lustre-OST0001: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x2c0000401:3 to 0x2c0000401:33) [11180.130289] Lustre: DEBUG MARKER: oleg211-server.virtnet: executing set_default_debug -1 all [11187.524495] Lustre: DEBUG MARKER: oleg211-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0001-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [11195.649831] Lustre: DEBUG MARKER: oleg211-server.virtnet: executing wait_import_state (FULL|IDLE) os[cp].lustre-OST0001-osc-MDT0001.ost_server_uuid 50 [11195.922384] Lustre: DEBUG MARKER: os[cp].lustre-OST0001-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [11296.224883] LustreError: lustre-OST0001-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [11296.231906] Lustre: lustre-OST0001-osc-MDT0000: Connection to lustre-OST0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [11296.273771] Lustre: lustre-OST0001: Not available for connect from 0@lo (stopping) [11296.280383] Lustre: Skipped 1 previous similar message [11300.828775] Lustre: server umount lustre-OST0001 complete [11301.350177] LustreError: 329696:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [11301.368322] LustreError: 329696:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 1 previous similar message [11306.464314] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [11306.473815] Lustre: Skipped 1 previous similar message [11306.478923] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [11306.482882] Lustre: Skipped 1 previous similar message [11311.590157] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [11311.603939] Lustre: Skipped 1 previous similar message [11316.714061] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [11316.732365] Lustre: Skipped 1 previous similar message [11320.287237] Lustre: lustre-OST0000 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [11320.491316] Lustre: server umount lustre-OST0000 complete [11324.904442] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [11324.921849] LustreError: Skipped 1 previous similar message [11324.930794] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [11324.970699] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [11326.944937] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [11330.875940] Lustre: server umount lustre-MDT0000 complete [11334.511479] LustreError: 327994:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1787443522 with bad export cookie 8798642530055403520 [11334.521905] LustreError: MGC192.168.202.111@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [11334.527509] LustreError: 327994:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [11334.925950] Lustre: server umount lustre-MDT0001 complete [11349.820712] Lustre: DEBUG MARKER: oleg211-server.virtnet: executing unload_modules_local [11353.218266] Key type lgssc unregistered [11353.513856] LNet: 332964:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [11353.518683] LNetError: 332964:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [11354.537220] LNet: Removed LNI 192.168.202.111@tcp [11355.527668] Key type .llcrypt unregistered [11355.530463] Key type ._llcrypt unregistered [11403.973378] Key type ._llcrypt registered [11403.978975] Key type .llcrypt registered [11404.176600] Lustre: DEBUG MARKER: oleg211-server.virtnet: executing load_modules_local [11405.856179] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [11405.905368] alg: No test for adler32 (adler32-zlib) [11407.007871] Lustre: Lustre: Build Version: 2.17.57_63_gaeb0ce0 [11407.312941] LNet: Added LNI 192.168.202.111@tcp [8/256/0/180] [11409.007207] Key type lgssc registered [11410.797979] Lustre: Echo OBD driver; http://www.lustre.org/ [11422.936949] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [11422.957966] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [11424.564627] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [11430.993744] Lustre: DEBUG MARKER: oleg211-server.virtnet: executing set_default_debug -1 all [11433.955912] LustreError: 335259:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [11433.981734] LustreError: 335259:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 2 previous similar messages [11439.073977] LustreError: 335260:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [11441.422884] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [11441.762145] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [11446.523937] Lustre: DEBUG MARKER: oleg211-server.virtnet: executing set_default_debug -1 all [11449.814464] Lustre: Modifying parameter general.debug_raw_pointers=Y in log params [11458.235344] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [11458.693444] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [11465.846901] LustreError: 336750:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [11466.227022] Lustre: DEBUG MARKER: oleg211-server.virtnet: executing set_default_debug -1 all [11468.913353] LustreError: 336751:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [11468.937517] LustreError: 336751:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 1 previous similar message [11468.944984] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:36 to 0x280000401:97) [11474.417633] LustreError: 336749:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [11474.427573] LustreError: 336749:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 1 previous similar message [11475.274337] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [11475.507741] Lustre: lustre-OST0001: Imperative Recovery not enabled, recovery window 60-180 [11480.579254] Lustre: lustre-OST0001: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x2c0000401:3 to 0x2c0000401:65) [11482.808936] Lustre: DEBUG MARKER: oleg211-server.virtnet: executing set_default_debug -1 all [11493.861927] Lustre: DEBUG MARKER: Using TIMEOUT=20 [11502.840550] Lustre: Setting parameter general.lod.*.mdt_hash=crush in log params [11516.390754] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [11516.395726] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [11516.421561] Lustre: Skipped 2 previous similar messages [11516.422936] Lustre: Skipped 1 previous similar message [11520.806226] Lustre: server umount lustre-MDT0000 complete [11521.527407] LustreError: 335990:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [11521.551736] LustreError: 335990:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 3 previous similar messages [11524.598367] LustreError: 338269:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1787443712 with bad export cookie 16561792922887064653 [11524.606426] LustreError: MGC192.168.202.111@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [11524.609183] LustreError: 338269:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 2 previous similar messages [11524.982191] Lustre: server umount lustre-MDT0001 complete [11539.278824] Lustre: server umount lustre-OST0000 complete [11542.509102] Lustre: 334522:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1787443714/real 1787443714] req@ffff9c91cfb8ad80 x1874270455139200/t0(0) o400->lustre-MDT0001-lwp-OST0001@0@lo:12/10 lens 224/224 e 0 to 1 dl 1787443730 ref 1 fl Rpc:XNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 [11542.555147] Lustre: lustre-MDT0001-lwp-OST0001: Connection to lustre-MDT0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [11542.579103] Lustre: Skipped 1 previous similar message [11543.605993] Lustre: server umount lustre-OST0001 complete [11564.074787] Lustre: DEBUG MARKER: oleg211-server.virtnet: executing load_modules_local [11573.462655] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [11573.941721] LustreError: 340522:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [11574.015813] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [11579.077910] Lustre: DEBUG MARKER: oleg211-server.virtnet: executing set_default_debug -1 all [11587.984328] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [11588.579129] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [11592.978513] Lustre: DEBUG MARKER: oleg211-server.virtnet: executing set_default_debug -1 all [11596.403627] Lustre: 341665:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify general/debug_raw_pointers=Y (mode = 0) failed: rc = -17 [11603.427938] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [11609.967659] Lustre: DEBUG MARKER: oleg211-server.virtnet: executing set_default_debug -1 all [11613.039982] LustreError: 342020:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [11613.049118] LustreError: 342020:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 2 previous similar messages [11619.011946] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [11619.314732] Lustre: lustre-OST0001: Imperative Recovery not enabled, recovery window 60-180 [11619.325956] Lustre: Skipped 1 previous similar message [11620.364354] Lustre: lustre-OST0001: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x2c0000401:3 to 0x2c0000401:97) [11620.367356] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:36 to 0x280000401:129) [11625.967088] Lustre: DEBUG MARKER: oleg211-server.virtnet: executing set_default_debug -1 all [11633.752226] Lustre: DEBUG MARKER: Using TIMEOUT=20 [11638.947905] Lustre: 343535:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify general/lod.*.mdt_hash=crush (mode = 0) failed: rc = -17 [11644.898450] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [11644.907577] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [11644.928597] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [11644.937098] Lustre: Skipped 2 previous similar messages [11645.932222] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [11645.937536] Lustre: Skipped 3 previous similar messages [11649.907972] Lustre: server umount lustre-MDT0000 complete [11654.088153] LustreError: 340503:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1787443842 with bad export cookie 16561792922887066354 [11654.094257] LustreError: MGC192.168.202.111@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [11654.105652] LustreError: 340503:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 2 previous similar messages [11654.659056] Lustre: server umount lustre-MDT0001 complete [11668.083052] Lustre: server umount lustre-OST0000 complete [11682.731755] Lustre: server umount lustre-OST0001 complete [11686.599070] Lustre: DEBUG MARKER: SKIP: conf-sanity test_114 skipping SLOW test 114 [11688.184266] Lustre: DEBUG MARKER: SKIP: conf-sanity test_115 skipping excluded test 115 [11689.932395] Lustre: DEBUG MARKER: == conf-sanity test 116: big size MDT support ============ 20:11:16 (1787443876) [11713.080389] Lustre: DEBUG MARKER: oleg211-server.virtnet: executing load_modules_local [11723.041209] loop0: detected capacity change from 0 to 1099511627776 [11723.541050] SGI XFS with ACLs, security attributes, quota, no debug enabled [11723.549587] *** VALIDATE xfs *** [11723.558197] XFS (loop0): Mounting V5 Filesystem [11723.575902] XFS (loop0): Ending clean mount [11724.739280] loop1: detected capacity change from 0 to 18691697672192 [11837.618686] Lustre: DEBUG MARKER: oleg211-server.virtnet: executing set_hostid [11848.533436] Lustre: DEBUG MARKER: oleg211-server.virtnet: executing load_modules_local [11881.050446] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [11881.278779] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [11881.303282] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [11881.368739] Lustre: lustre-MDT0000: new disk, initializing [11881.445641] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [11881.458715] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [11885.683707] Lustre: DEBUG MARKER: oleg211-server.virtnet: executing set_default_debug -1 all [11897.233469] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [11897.351978] Lustre: 348672:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [11897.392311] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [11897.395075] Lustre: Skipped 1 previous similar message [11897.436256] Lustre: lustre-MDT0001: new disk, initializing [11897.543131] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [11897.559100] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [11903.036660] Lustre: DEBUG MARKER: oleg211-server.virtnet: executing set_default_debug -1 all [11912.309365] Lustre: DEBUG MARKER: oleg211-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [11918.452953] Lustre: DEBUG MARKER: oleg211-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [11924.801282] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [11925.157982] Lustre: lustre-OST0000: new disk, initializing [11925.164616] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [11925.182338] Lustre: 350638:0:(osd_compat.c:1353:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [11926.344970] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [11926.359945] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [11926.500246] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [11932.546320] Lustre: DEBUG MARKER: oleg211-server.virtnet: executing set_default_debug -1 all [11942.071167] Lustre: DEBUG MARKER: oleg211-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [11946.976160] Lustre: DEBUG MARKER: oleg211-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 [11947.237799] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [11953.922943] Lustre: DEBUG MARKER: oleg211-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid 50 [11954.214517] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [11957.235726] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [11957.247402] Lustre: Skipped 3 previous similar messages [11957.252766] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [11957.257411] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [11957.286659] LustreError: Skipped 1 previous similar message [11962.340394] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [11962.355887] Lustre: Skipped 2 previous similar messages [11962.580050] Lustre: server umount lustre-OST0000 complete [11967.459972] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [11967.480316] Lustre: Skipped 2 previous similar messages [11972.584714] LustreError: 348682:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [11972.619675] LustreError: 348682:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 5 previous similar messages [11972.878957] Lustre: server umount lustre-MDT0000 complete [11976.458337] LustreError: 348662:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1787444164 with bad export cookie 16561792922887067908 [11976.475684] LustreError: MGC192.168.202.111@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [11976.476977] LustreError: 348662:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 3 previous similar messages [11976.983260] Lustre: server umount lustre-MDT0001 complete [11985.553180] XFS (loop0): Unmounting Filesystem [11991.045844] Lustre: DEBUG MARKER: == conf-sanity test 117: lctl get_param return errors properly ========================================================== 20:16:17 (1787444177) [11997.803869] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [11998.539978] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [11998.548498] Lustre: Skipped 2 previous similar messages [12002.856525] Lustre: DEBUG MARKER: oleg211-server.virtnet: executing set_default_debug -1 all [12011.025538] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [12015.873685] Lustre: DEBUG MARKER: oleg211-server.virtnet: executing set_default_debug -1 all [12022.602547] Lustre: DEBUG MARKER: oleg211-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [12028.010448] Lustre: DEBUG MARKER: oleg211-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [12034.196294] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [12041.366934] Lustre: DEBUG MARKER: oleg211-server.virtnet: executing set_default_debug -1 all [12048.797913] Lustre: DEBUG MARKER: oleg211-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [12059.618486] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [12059.623715] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [12059.634429] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [12059.637081] Lustre: Skipped 1 previous similar message [12059.655580] Lustre: Skipped 3 previous similar messages [12063.999049] Lustre: server umount lustre-OST0000 complete [12075.262971] Lustre: server umount lustre-MDT0000 complete [12079.625095] LustreError: 353179:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1787444267 with bad export cookie 16561792922887068790 [12079.629502] LustreError: MGC192.168.202.111@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [12079.649869] LustreError: 353179:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [12080.181496] Lustre: server umount lustre-MDT0001 complete [12092.641586] Lustre: DEBUG MARKER: oleg211-server.virtnet: executing unload_modules_local [12096.739825] Key type lgssc unregistered [12097.194855] LNet: 356829:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [12097.210447] LNetError: 356829:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [12097.229814] LNet: Removed LNI 192.168.202.111@tcp [12098.420195] Key type .llcrypt unregistered [12098.421568] Key type ._llcrypt unregistered [12126.985618] Key type ._llcrypt registered [12126.987337] Key type .llcrypt registered [12127.154300] Lustre: DEBUG MARKER: oleg211-server.virtnet: executing load_modules_local [12128.692332] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [12128.884874] alg: No test for adler32 (adler32-zlib) [12130.241875] Lustre: Lustre: Build Version: 2.17.57_63_gaeb0ce0 [12130.711300] LNet: Added LNI 192.168.202.111@tcp [8/256/0/180] [12132.495771] Key type lgssc registered [12133.981603] Lustre: Echo OBD driver; http://www.lustre.org/ [12142.602421] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [12142.617356] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [12144.179534] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [12148.677293] Lustre: DEBUG MARKER: oleg211-server.virtnet: executing set_default_debug -1 all [12154.362197] LustreError: 358396:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [12154.379501] LustreError: 358396:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 2 previous similar messages [12157.986359] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [12158.461885] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [12163.780960] Lustre: DEBUG MARKER: oleg211-server.virtnet: executing set_default_debug -1 all [12173.663523] Lustre: DEBUG MARKER: oleg211-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [12179.783568] Lustre: DEBUG MARKER: oleg211-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [12186.735120] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [12187.138747] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [12193.404442] Lustre: DEBUG MARKER: oleg211-server.virtnet: executing set_default_debug -1 all [12200.976861] Lustre: DEBUG MARKER: oleg211-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [12213.270473] Lustre: lustre-MDT0001: Not available for connect from 192.168.202.11@tcp (stopping) [12213.732865] Lustre: lustre-MDT0001-osp-MDT0000: Connection to lustre-MDT0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [12217.913277] Lustre: server umount lustre-MDT0001 complete [12218.364327] LustreError: 358392:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 192.168.202.11@tcp (no target). If you are running an HA pair check that the target is mounted on the other server. [12223.497129] LustreError: 358393:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 192.168.202.11@tcp (no target). If you are running an HA pair check that the target is mounted on the other server. [12223.512673] LustreError: 358393:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 2 previous similar messages [12228.614511] LustreError: 358393:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 192.168.202.11@tcp (no target). If you are running an HA pair check that the target is mounted on the other server. [12228.662248] LustreError: 358393:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 2 previous similar messages [12229.188367] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [12229.343442] Lustre: MGS: Regenerating lustre-MDT0001 log by user request: rc = 0 [12229.349869] Lustre: Found index 1 for lustre-MDT0001, updating log [12229.440787] Lustre: 358386:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [12229.872929] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [12234.308642] Lustre: DEBUG MARKER: oleg211-server.virtnet: executing set_default_debug -1 all [12235.261772] Lustre: lustre-MDT0001-lwp-OST0000: Connection restored to 0@lo (at 0@lo) [12236.293318] LustreError: 361953:0:(obd_mount.c:234:lustre_start_simple()) lustre-MDT0001-lwp-OST0000 attach error -17 [12236.314523] LustreError: 361953:0:(tgt_mount.c:675:lustre_lwp_setup()) lustre-MDT0001-lwp-OST0000: setup up failed: rc -17 [12236.330538] LustreError: 358389:0:(mgc_request.c:614:do_requeue()) failed processing log: -17 [12236.412168] LustreError: 361963:0:(obd_config.c:2102:class_config_llog_handler()) MGC192.168.202.111@tcp: cfg command failed: rc = -17 [12236.422069] Lustre: cmd=cf001 0:lustre-MDT0001-osp-MDT0000 1:osp 2:lustre-MDT0000-mdtlov_UUID [12236.422069] [12243.990078] Lustre: lustre-MDT0001: Not available for connect from 192.168.202.11@tcp (stopping) [12244.000416] Lustre: Skipped 2 previous similar messages [12245.480889] Lustre: lustre-MDT0001-osp-MDT0000: Connection to lustre-MDT0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [12245.482219] LustreError: lustre-MDT0001-osp-MDT0000: operation mds_statfs to node 0@lo failed: rc = -107 [12245.493055] Lustre: Skipped 1 previous similar message [12245.509349] Lustre: lustre-MDT0001: Not available for connect from 0@lo (stopping) [12245.520305] Lustre: Skipped 1 previous similar message [12249.092619] Lustre: lustre-MDT0001: Not available for connect from 192.168.202.11@tcp (stopping) [12249.685275] Lustre: server umount lustre-MDT0001 complete [12250.595260] LustreError: 360838:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [12250.605888] LustreError: 360838:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 3 previous similar messages [12259.334685] LustreError: 358391:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 192.168.202.11@tcp (no target). If you are running an HA pair check that the target is mounted on the other server. [12259.369667] LustreError: 358391:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 3 previous similar messages [12260.523602] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [12260.628542] Lustre: MGS: Regenerating lustre-MDT0001 log by user request: rc = 0 [12260.632206] Lustre: Found index 1 for lustre-MDT0001, updating log [12260.711587] Lustre: 358385:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [12260.849279] Lustre: lustre-MDT0001: Not available for connect from 0@lo (not set up) [12261.204872] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [12266.062527] Lustre: DEBUG MARKER: oleg211-server.virtnet: executing set_default_debug -1 all [12266.480375] Lustre: lustre-MDT0001-lwp-OST0000: Connection restored to 0@lo (at 0@lo) [12269.050687] LustreError: 363061:0:(obd_mount.c:234:lustre_start_simple()) lustre-MDT0001-lwp-OST0000 attach error -17 [12269.061092] LustreError: 363061:0:(tgt_mount.c:675:lustre_lwp_setup()) lustre-MDT0001-lwp-OST0000: setup up failed: rc -17 [12269.077263] LustreError: 358389:0:(mgc_request.c:614:do_requeue()) failed processing log: -17 [12269.089532] LustreError: 358389:0:(mgc_request.c:614:do_requeue()) Skipped 1 previous similar message [12269.201594] LustreError: 363072:0:(obd_config.c:761:class_setup()) osp: device 8 already setup: rc = -17 [12269.211424] LustreError: 363072:0:(obd_config.c:866:class_detach()) OBD device 8 still set up [12269.218684] LustreError: 363072:0:(obd_config.c:2102:class_config_llog_handler()) MGC192.168.202.111@tcp: cfg command failed: rc = -17 [12269.232942] Lustre: cmd=cf003 0:lustre-MDT0001-osp-MDT0000 1:lustre-MDT0001_UUID 2:192.168.202.111@tcp [12269.232942] [12573.664894] Lustre: lustre-MDT0001-lwp-OST0000: Connection to lustre-MDT0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [12573.668779] LustreError: lustre-MDT0001-osp-MDT0000: operation mds_statfs to node 0@lo failed: rc = -107 [12573.689074] Lustre: Skipped 1 previous similar message [12573.690239] Lustre: lustre-MDT0001: Not available for connect from 0@lo (stopping) [12573.690246] Lustre: Skipped 1 previous similar message [12579.454103] Lustre: server umount lustre-MDT0001 complete [12581.883027] LustreError: 358393:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 192.168.202.11@tcp (no target). If you are running an HA pair check that the target is mounted on the other server. [12590.762229] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [12590.903063] Lustre: MGS: Regenerating lustre-MDT0001 log by user request: rc = 0 [12590.906503] Lustre: Found index 1 for lustre-MDT0001, updating log [12590.948128] Lustre: 358386:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [12591.306271] Lustre: lustre-MDT0001: Imperative Recovery enabled, recovery window shrunk from 60-180 down to 60-180 [12596.246586] LustreError: 371375:0:(obd_mount.c:234:lustre_start_simple()) lustre-MDT0001-lwp-OST0000 attach error -17 [12596.261435] LustreError: 371375:0:(tgt_mount.c:675:lustre_lwp_setup()) lustre-MDT0001-lwp-OST0000: setup up failed: rc -17 [12596.274978] LustreError: 358389:0:(mgc_request.c:614:do_requeue()) failed processing log: -17 [12596.290615] LustreError: 358389:0:(mgc_request.c:614:do_requeue()) Skipped 1 previous similar message [12596.386597] LustreError: 371394:0:(lod_lov.c:208:lod_add_device()) lustre-MDT0001-osp-MDT0000: cannot connect to next dev lustre-MDT0001_UUID (-114) [12596.399652] LustreError: 371394:0:(obd_config.c:2102:class_config_llog_handler()) MGC192.168.202.111@tcp: cfg command failed: rc = -114 [12596.406308] Lustre: cmd=cf014 0:lustre-MDT0000-mdtlov 1:lustre-MDT0001_UUID 2:1 3:1 [12596.406308] [12596.427099] Lustre: DEBUG MARKER: oleg211-server.virtnet: executing set_default_debug -1 all [12596.745229] Lustre: lustre-MDT0001-lwp-OST0000: Connection restored to 0@lo (at 0@lo) [12919.275359] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [12919.285802] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [12919.327560] Lustre: Skipped 1 previous similar message [12919.337580] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [12919.347133] Lustre: Skipped 4 previous similar messages [12924.759275] Lustre: server umount lustre-MDT0000 complete [12928.360940] LustreError: 360241:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1787445116 with bad export cookie 11270997969240498366 [12928.365545] LustreError: MGC192.168.202.111@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [12928.369418] LustreError: 360241:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 2 previous similar messages [12929.505647] Lustre: lustre-MDT0001-lwp-OST0000: Connection to lustre-MDT0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [12929.509344] LustreError: 358393:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [12929.518178] Lustre: Skipped 2 previous similar messages [12929.541207] LustreError: 358393:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 5 previous similar messages [12934.821916] Lustre: server umount lustre-MDT0001 complete [12939.526160] Lustre: server umount lustre-OST0000 complete [12949.186266] Lustre: DEBUG MARKER: oleg211-server.virtnet: executing set_hostid [12960.957393] Lustre: DEBUG MARKER: oleg211-server.virtnet: executing load_modules_local [13002.404479] Lustre: DEBUG MARKER: == conf-sanity test 120: cross-target rename should not create bad symlinks ========================================================== 20:33:08 (1787445188) [13010.305428] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [13010.559662] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [13010.582365] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [13010.674533] Lustre: lustre-MDT0000: new disk, initializing [13010.746542] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [13010.759298] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [13015.471702] Lustre: DEBUG MARKER: oleg211-server.virtnet: executing set_default_debug -1 all [13028.134550] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [13028.306221] Lustre: 381824:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [13028.375733] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [13028.382363] Lustre: Skipped 1 previous similar message [13028.480289] Lustre: lustre-MDT0001: new disk, initializing [13028.580273] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [13028.634720] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [13028.643574] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [13033.054897] Lustre: DEBUG MARKER: oleg211-server.virtnet: executing set_default_debug -1 all [13043.912513] Lustre: DEBUG MARKER: oleg211-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [13050.627495] Lustre: DEBUG MARKER: oleg211-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [13058.692443] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [13059.153542] Lustre: lustre-OST0000: new disk, initializing [13059.158448] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [13059.181416] Lustre: 383797:0:(osd_compat.c:1353:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [13059.398420] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [13065.849410] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [13065.861875] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [13066.033213] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [13067.776715] Lustre: DEBUG MARKER: oleg211-server.virtnet: executing set_default_debug -1 all [13078.683669] Lustre: DEBUG MARKER: oleg211-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [13087.728573] LustreError: lustre-MDT0000-osp-MDT0001: operation out_update to node 0@lo failed: rc = -107 [13087.739512] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [13087.757711] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [13087.759743] Lustre: Skipped 7 previous similar messages [13089.295700] Lustre: server umount lustre-MDT0000 complete [13091.810868] LustreError: 381837:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [13091.845519] LustreError: 381837:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 3 previous similar messages [13095.053966] LustreError: 381819:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1787445283 with bad export cookie 11270997969240500326 [13095.063637] LustreError: MGC192.168.202.111@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [13095.065405] LustreError: 381819:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 2 previous similar messages [13095.088704] LustreError: 382635:0:(osp_object.c:618:osp_attr_get()) lustre-MDT0000-osp-MDT0001: osp_attr_get update error [0x200000401:0x2:0x0]: rc = -5 [13095.640981] Lustre: server umount lustre-MDT0001 complete [13110.108192] Lustre: server umount lustre-OST0000 complete [13143.496892] Lustre: DEBUG MARKER: oleg211-server.virtnet: executing set_hostid [13155.078826] Lustre: DEBUG MARKER: oleg211-server.virtnet: executing load_modules_local [13189.958975] Lustre: DEBUG MARKER: == conf-sanity test complete, duration 12847 sec ========= 20:36:15 (1787445375) [13191.913625] Lustre: DEBUG MARKER: === conf-sanity: start cleanup 20:36:17 (1787445377) === [13195.988940] Lustre: DEBUG MARKER: === conf-sanity: finish cleanup 20:36:21 (1787445381) === [13242.224957] Lustre: DEBUG MARKER: oleg211-server.virtnet: executing unload_modules_local [13245.440343] Key type lgssc unregistered [13245.930778] LNet: 390166:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [13245.937408] LNetError: 390166:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [13245.957769] LNet: Removed LNI 192.168.202.111@tcp [13246.993660] Key type .llcrypt unregistered [13246.996060] Key type ._llcrypt unregistered