[ 0.000000] Linux version 4.18.0rh8.10-debug (green@maintenance) (gcc version 8.5.0 20210514 (Red Hat 8.5.0-26) (GCC)) #2 SMP Mon Jul 14 01:24:22 EDT 2025 [ 0.000000] Command line: rd.shell root=nbd:192.168.200.253:rocky8.10:ext4:ro:-p,-b4096 ro crashkernel=256M panic=1 nomodeset ipmtu=9000 ip=dhcp rd.neednet=1 init_on_free=off mitigations=off console=ttyS1,115200 audit=0 [ 0.000000] x86/fpu: Supporting XSAVE feature 0x001: 'x87 floating point registers' [ 0.000000] x86/fpu: Supporting XSAVE feature 0x002: 'SSE registers' [ 0.000000] x86/fpu: Supporting XSAVE feature 0x004: 'AVX registers' [ 0.000000] x86/fpu: xstate_offset[2]: 576, xstate_sizes[2]: 256 [ 0.000000] x86/fpu: Enabled xstate features 0x7, context size is 832 bytes, using 'standard' format. [ 0.000000] signal: max sigframe size: 1776 [ 0.000000] BIOS-provided physical RAM map: [ 0.000000] BIOS-e820: [mem 0x0000000000000000-0x000000000009fbff] usable [ 0.000000] BIOS-e820: [mem 0x000000000009fc00-0x000000000009ffff] reserved [ 0.000000] BIOS-e820: [mem 0x00000000000f0000-0x00000000000fffff] reserved [ 0.000000] BIOS-e820: [mem 0x0000000000100000-0x00000000bffcdfff] usable [ 0.000000] BIOS-e820: [mem 0x00000000bffce000-0x00000000bfffffff] reserved [ 0.000000] BIOS-e820: [mem 0x00000000feffc000-0x00000000feffffff] reserved [ 0.000000] BIOS-e820: [mem 0x00000000fffc0000-0x00000000ffffffff] reserved [ 0.000000] BIOS-e820: [mem 0x0000000100000000-0x0000000146dfffff] usable [ 0.000000] NX (Execute Disable) protection: active [ 0.000000] SMBIOS 2.8 present. [ 0.000000] DMI: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.17.0-10.fc44 06/10/2025 [ 0.000000] Hypervisor detected: KVM [ 0.000000] kvm-clock: Using msrs 4b564d01 and 4b564d00 [ 0.000000] kvm-clock: using sched offset of 478986515 cycles [ 0.000000] clocksource: kvm-clock: mask: 0xffffffffffffffff max_cycles: 0x1cd42e4dffb, max_idle_ns: 881590591483 ns [ 0.000000] tsc: Detected 2399.998 MHz processor [ 0.000000] last_pfn = 0x146e00 max_arch_pfn = 0x400000000 [ 0.000000] x86/PAT: Configuration [0-7]: WB WC UC- UC WB WP UC- WT [ 0.000000] last_pfn = 0xbffce max_arch_pfn = 0x400000000 [ 0.000000] found SMP MP-table at [mem 0x000f54b0-0x000f54bf] [ 0.000000] RAMDISK: [mem 0xbcc54000-0xbffbffff] [ 0.000000] ACPI: Early table checksum verification disabled [ 0.000000] ACPI: RSDP 0x00000000000F52D0 000014 (v00 BOCHS ) [ 0.000000] ACPI: RSDT 0x00000000BFFE247C 000034 (v01 BOCHS BXPC 00000001 BXPC 00000001) [ 0.000000] ACPI: FACP 0x00000000BFFE2318 000074 (v01 BOCHS BXPC 00000001 BXPC 00000001) [ 0.000000] ACPI: DSDT 0x00000000BFFE0040 0022D8 (v01 BOCHS BXPC 00000001 BXPC 00000001) [ 0.000000] ACPI: FACS 0x00000000BFFE0000 000040 [ 0.000000] ACPI: APIC 0x00000000BFFE238C 000090 (v03 BOCHS BXPC 00000001 BXPC 00000001) [ 0.000000] ACPI: HPET 0x00000000BFFE241C 000038 (v01 BOCHS BXPC 00000001 BXPC 00000001) [ 0.000000] ACPI: WAET 0x00000000BFFE2454 000028 (v01 BOCHS BXPC 00000001 BXPC 00000001) [ 0.000000] ACPI: Reserving FACP table memory at [mem 0xbffe2318-0xbffe238b] [ 0.000000] ACPI: Reserving DSDT table memory at [mem 0xbffe0040-0xbffe2317] [ 0.000000] ACPI: Reserving FACS table memory at [mem 0xbffe0000-0xbffe003f] [ 0.000000] ACPI: Reserving APIC table memory at [mem 0xbffe238c-0xbffe241b] [ 0.000000] ACPI: Reserving HPET table memory at [mem 0xbffe241c-0xbffe2453] [ 0.000000] ACPI: Reserving WAET table memory at [mem 0xbffe2454-0xbffe247b] [ 0.000000] No NUMA configuration found [ 0.000000] Faking a node at [mem 0x0000000000000000-0x0000000146dfffff] [ 0.000000] NODE_DATA(0) allocated [mem 0x1465a3000-0x1465cdfff] [ 0.000000] Reserving 256MB of memory at 2752MB for crashkernel (System RAM: 4205MB) [ 0.000000] Zone ranges: [ 0.000000] DMA [mem 0x0000000000001000-0x0000000000ffffff] [ 0.000000] DMA32 [mem 0x0000000001000000-0x00000000ffffffff] [ 0.000000] Normal [mem 0x0000000100000000-0x0000000146dfffff] [ 0.000000] Device empty [ 0.000000] Movable zone start for each node [ 0.000000] Early memory node ranges [ 0.000000] node 0: [mem 0x0000000000001000-0x000000000009efff] [ 0.000000] node 0: [mem 0x0000000000100000-0x00000000bffcdfff] [ 0.000000] node 0: [mem 0x0000000100000000-0x0000000146dfffff] [ 0.000000] Zeroed struct page in unavailable ranges: 4756 pages [ 0.000000] Initmem setup node 0 [mem 0x0000000000001000-0x0000000146dfffff] [ 0.000000] ACPI: PM-Timer IO Port: 0x608 [ 0.000000] ACPI: LAPIC_NMI (acpi_id[0xff] dfl dfl lint[0x1]) [ 0.000000] IOAPIC[0]: apic_id 0, version 17, address 0xfec00000, GSI 0-23 [ 0.000000] ACPI: INT_SRC_OVR (bus 0 bus_irq 0 global_irq 2 dfl dfl) [ 0.000000] ACPI: INT_SRC_OVR (bus 0 bus_irq 5 global_irq 5 high level) [ 0.000000] ACPI: INT_SRC_OVR (bus 0 bus_irq 9 global_irq 9 high level) [ 0.000000] ACPI: INT_SRC_OVR (bus 0 bus_irq 10 global_irq 10 high level) [ 0.000000] ACPI: INT_SRC_OVR (bus 0 bus_irq 11 global_irq 11 high level) [ 0.000000] Using ACPI (MADT) for SMP configuration information [ 0.000000] ACPI: HPET id: 0x8086a201 base: 0xfed00000 [ 0.000000] TSC deadline timer available [ 0.000000] smpboot: Allowing 4 CPUs, 0 hotplug CPUs [ 0.000000] kvm-guest: KVM setup pv remote TLB flush [ 0.000000] kvm-guest: setup PV sched yield [ 0.000000] PM: Registered nosave memory: [mem 0x00000000-0x00000fff] [ 0.000000] PM: Registered nosave memory: [mem 0x0009f000-0x0009ffff] [ 0.000000] PM: Registered nosave memory: [mem 0x000a0000-0x000effff] [ 0.000000] PM: Registered nosave memory: [mem 0x000f0000-0x000fffff] [ 0.000000] PM: Registered nosave memory: [mem 0xbffce000-0xbfffffff] [ 0.000000] PM: Registered nosave memory: [mem 0xc0000000-0xfeffbfff] [ 0.000000] PM: Registered nosave memory: [mem 0xfeffc000-0xfeffffff] [ 0.000000] PM: Registered nosave memory: [mem 0xff000000-0xfffbffff] [ 0.000000] PM: Registered nosave memory: [mem 0xfffc0000-0xffffffff] [ 0.000000] [mem 0xc0000000-0xfeffbfff] available for PCI devices [ 0.000000] Booting paravirtualized kernel on KVM [ 0.000000] clocksource: refined-jiffies: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 1910969940391419 ns [ 0.000000] setup_percpu: NR_CPUS:8192 nr_cpumask_bits:4 nr_cpu_ids:4 nr_node_ids:1 [ 0.000000] percpu: Embedded 63 pages/cpu s221184 r8192 d28672 u524288 [ 0.000000] kvm-guest: PV spinlocks enabled [ 0.000000] PV qspinlock hash table entries: 256 (order: 0, 4096 bytes, linear) [ 0.000000] Built 1 zonelists, mobility grouping on. Total pages: 1059606 [ 0.000000] Policy zone: Normal [ 0.000000] Kernel command line: rd.shell root=nbd:192.168.200.253:rocky8.10:ext4:ro:-p,-b4096 ro crashkernel=256M panic=1 nomodeset ipmtu=9000 ip=dhcp rd.neednet=1 init_on_free=off mitigations=off console=ttyS1,115200 audit=0 [ 0.000000] Specific versions of hardware are certified with Red Hat Enterprise Linux 8. Please see the list of hardware certified with Red Hat Enterprise Linux 8 at https://catalog.redhat.com. [ 0.000000] audit: disabled (until reboot) [ 0.000000] software IO TLB: area num 4. [ 0.000000] Memory: 2829652K/4306352K available (18435K kernel code, 11221K rwdata, 7248K rodata, 2908K init, 18040K bss, 524580K reserved, 0K cma-reserved) [ 0.000000] SLUB: HWalign=64, Order=0-3, MinObjects=0, CPUs=4, Nodes=1 [ 0.000000] kmemleak: Kernel memory leak detector disabled [ 0.000000] ftrace: allocating 41240 entries in 162 pages [ 0.000000] ftrace: allocated 162 pages with 3 groups [ 0.000000] rcu: Hierarchical RCU implementation. [ 0.000000] rcu: RCU event tracing is enabled. [ 0.000000] rcu: RCU restricting CPUs from NR_CPUS=8192 to nr_cpu_ids=4. [ 0.000000] rcu: RCU callback double-/use-after-free debug enabled. [ 0.000000] Rude variant of Tasks RCU enabled. [ 0.000000] Tracing variant of Tasks RCU enabled. [ 0.000000] rcu: RCU calculated value of scheduler-enlistment delay is 100 jiffies. [ 0.000000] rcu: Adjusting geometry for rcu_fanout_leaf=16, nr_cpu_ids=4 [ 0.000000] NR_IRQS: 524544, nr_irqs: 456, preallocated irqs: 16 [ 0.000000] random: get_random_bytes called from start_kernel+0x622/0x9a8 with crng_init=0 [ 0.001000] Console: colour *CGA 80x25 [ 0.001000] printk: console [ttyS1] enabled [ 0.001000] ACPI: Core revision 20220331 [ 0.001000] clocksource: hpet: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 19112604467 ns [ 0.001011] APIC: Switch to symmetric I/O mode setup [ 0.003268] x2apic enabled [ 0.004006] Switched APIC routing to physical x2apic. [ 0.005016] kvm-guest: setup PV IPIs [ 0.008281] ..TIMER: vector=0x30 apic1=0 pin1=2 apic2=-1 pin2=-1 [ 0.009000] clocksource: tsc-early: mask: 0xffffffffffffffff max_cycles: 0x229835b7123, max_idle_ns: 440795242976 ns [ 0.009020] Calibrating delay loop (skipped) preset value.. 4799.99 BogoMIPS (lpj=2399998) [ 0.010010] pid_max: default: 32768 minimum: 301 [ 0.011101] LSM: Security Framework initializing [ 0.012035] Yama: becoming mindful. [ 0.013029] SELinux: Initializing. [ 0.015002] *** VALIDATE selinux *** [ 0.023439] Dentry cache hash table entries: 1048576 (order: 11, 8388608 bytes, vmalloc) [ 0.028385] Inode-cache hash table entries: 524288 (order: 10, 4194304 bytes, vmalloc) [ 0.029172] Mount-cache hash table entries: 16384 (order: 5, 131072 bytes, vmalloc) [ 0.030125] Mountpoint-cache hash table entries: 16384 (order: 5, 131072 bytes, vmalloc) [ 0.032087] *** VALIDATE tmpfs *** [ 0.034064] *** VALIDATE proc *** [ 0.035281] *** VALIDATE cgroup *** [ 0.036013] *** VALIDATE cgroup2 *** [ 0.037272] x86/cpu: User Mode Instruction Prevention (UMIP) activated [ 0.038157] Last level iTLB entries: 4KB 0, 2MB 0, 4MB 0 [ 0.039008] Last level dTLB entries: 4KB 0, 2MB 0, 4MB 0, 1GB 0 [ 0.041003] Spectre V2 : User space: Vulnerable [ 0.042000] Speculative Store Bypass: Vulnerable [ 0.044588] debug: unmapping init [mem 0xffffffff87259000-0xffffffff87260fff] [ 0.046951] smpboot: CPU0: Intel(R) Xeon(R) CPU E5-2695 v2 @ 2.40GHz (family: 0x6, model: 0x3e, stepping: 0x4) [ 0.047687] Performance Events: IvyBridge events, full-width counters, Intel PMU driver. [ 0.048021] ... version: 2 [ 0.049013] ... bit width: 48 [ 0.050011] ... generic registers: 4 [ 0.051013] ... value mask: 0000ffffffffffff [ 0.052015] ... max period: 00007fffffffffff [ 0.053014] ... fixed-purpose events: 3 [ 0.054009] ... event mask: 000000070000000f [ 0.055254] rcu: Hierarchical SRCU implementation. [ 0.057361] smp: Bringing up secondary CPUs ... [ 0.058605] x86: Booting SMP configuration: [ 0.059025] .... node #0, CPUs: #1 #2 #3 [ 0.062376] smp: Brought up 1 node, 4 CPUs [ 0.064013] smpboot: Max logical packages: 1 [ 0.064918] smpboot: Total of 4 processors activated (19199.98 BogoMIPS) [ 0.228094] node 0 deferred pages initialised in 160ms [ 0.232015] devtmpfs: initialized [ 0.233241] x86/mm: Memory block size: 128MB [ 0.235672] gcov: version magic: 0x41383552 [ 0.238292] clocksource: jiffies: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 1911260446275000 ns [ 0.239077] futex hash table entries: 1024 (order: 4, 65536 bytes, vmalloc) [ 0.240251] pinctrl core: initialized pinctrl subsystem [ 0.241183] [ 0.241719] ************************************************************* [ 0.242011] ** NOTICE NOTICE NOTICE NOTICE NOTICE NOTICE NOTICE ** [ 0.243014] ** ** [ 0.244012] ** IOMMU DebugFS SUPPORT HAS BEEN ENABLED IN THIS KERNEL ** [ 0.245012] ** ** [ 0.246015] ** This means that this kernel is built to expose internal ** [ 0.247013] ** IOMMU data structures, which may compromise security on ** [ 0.248012] ** your system. ** [ 0.249011] ** ** [ 0.250011] ** If you see this message and you are not debugging the ** [ 0.251010] ** kernel, report this immediately to your vendor! ** [ 0.252011] ** ** [ 0.253010] ** NOTICE NOTICE NOTICE NOTICE NOTICE NOTICE NOTICE ** [ 0.254010] ************************************************************* [ 0.255708] NET: Registered protocol family 16 [ 0.257426] DMA: preallocated 512 KiB GFP_KERNEL pool for atomic allocations [ 0.260062] DMA: preallocated 512 KiB GFP_KERNEL|GFP_DMA pool for atomic allocations [ 0.263060] DMA: preallocated 512 KiB GFP_KERNEL|GFP_DMA32 pool for atomic allocations [ 0.267135] cpuidle: using governor menu [ 0.270111] acpiphp: ACPI Hot Plug PCI Controller Driver version: 0.5 [ 0.272516] PCI: Using configuration type 1 for base access [ 0.275117] core: PMU erratum BJ122, BV98, HSD29 worked around, HT is on [ 0.284271] HugeTLB registered 1.00 GiB page size, pre-allocated 0 pages [ 0.286015] HugeTLB registered 2.00 MiB page size, pre-allocated 0 pages [ 0.289137] cryptd: max_cpu_qlen set to 1000 [ 0.292233] ACPI: Added _OSI(Module Device) [ 0.294019] ACPI: Added _OSI(Processor Device) [ 0.295011] ACPI: Added _OSI(3.0 _SCP Extensions) [ 0.297014] ACPI: Added _OSI(Processor Aggregator Device) [ 0.302060] ACPI: 1 ACPI AML tables successfully acquired and loaded [ 0.308325] ACPI: Interpreter enabled [ 0.309057] ACPI: PM: (supports S0 S3 S4 S5) [ 0.311013] ACPI: Using IOAPIC for interrupt routing [ 0.312095] PCI: Using host bridge windows from ACPI; if necessary, use "pci=nocrs" and report a bug [ 0.315373] ACPI: Enabled 2 GPEs in block 00 to 0F [ 0.325677] ACPI: PCI Root Bridge [PCI0] (domain 0000 [bus 00-ff]) [ 0.328070] acpi PNP0A03:00: _OSC: OS supports [ASPM ClockPM Segments MSI HPX-Type3] [ 0.331029] acpi PNP0A03:00: _OSC: not requesting OS control; OS requires [ExtendedConfig ASPM ClockPM MSI] [ 0.335108] acpi PNP0A03:00: fail to add MMCONFIG information, can't access extended PCI configuration space under this bridge. [ 0.343000] acpiphp: Slot [2] registered [ 0.343000] acpiphp: Slot [5] registered [ 0.346241] acpiphp: Slot [6] registered [ 0.347185] acpiphp: Slot [7] registered [ 0.349199] acpiphp: Slot [8] registered [ 0.351177] acpiphp: Slot [9] registered [ 0.352189] acpiphp: Slot [10] registered [ 0.353162] acpiphp: Slot [3] registered [ 0.355172] acpiphp: Slot [4] registered [ 0.356124] acpiphp: Slot [11] registered [ 0.358133] acpiphp: Slot [12] registered [ 0.359108] acpiphp: Slot [13] registered [ 0.361110] acpiphp: Slot [14] registered [ 0.362115] acpiphp: Slot [15] registered [ 0.364105] acpiphp: Slot [16] registered [ 0.366088] acpiphp: Slot [17] registered [ 0.367086] acpiphp: Slot [18] registered [ 0.368070] acpiphp: Slot [19] registered [ 0.369051] acpiphp: Slot [20] registered [ 0.370075] acpiphp: Slot [21] registered [ 0.371120] acpiphp: Slot [22] registered [ 0.373108] acpiphp: Slot [23] registered [ 0.374056] acpiphp: Slot [24] registered [ 0.374902] acpiphp: Slot [25] registered [ 0.376101] acpiphp: Slot [26] registered [ 0.378105] acpiphp: Slot [27] registered [ 0.379030] acpiphp: Slot [28] registered [ 0.379829] acpiphp: Slot [29] registered [ 0.381085] acpiphp: Slot [30] registered [ 0.382092] acpiphp: Slot [31] registered [ 0.384061] PCI host bridge to bus 0000:00 [ 0.385017] pci_bus 0000:00: root bus resource [io 0x0000-0x0cf7 window] [ 0.387032] pci_bus 0000:00: root bus resource [io 0x0d00-0xffff window] [ 0.390022] pci_bus 0000:00: root bus resource [mem 0x000a0000-0x000bffff window] [ 0.393023] pci_bus 0000:00: root bus resource [mem 0xc0000000-0xfebfffff window] [ 0.395037] pci_bus 0000:00: root bus resource [mem 0xe0000000000-0xe007fffffff window] [ 0.398042] pci_bus 0000:00: root bus resource [bus 00-ff] [ 0.400216] pci 0000:00:00.0: [8086:1237] type 00 class 0x060000 [ 0.403127] pci 0000:00:01.0: [8086:7000] type 00 class 0x060100 [ 0.407424] pci 0000:00:01.1: [8086:7010] type 00 class 0x010180 [ 0.416699] pci 0000:00:01.1: reg 0x20: [io 0xc320-0xc32f] [ 0.421840] pci 0000:00:01.1: legacy IDE quirk: reg 0x10: [io 0x01f0-0x01f7] [ 0.425031] pci 0000:00:01.1: legacy IDE quirk: reg 0x14: [io 0x03f6] [ 0.427031] pci 0000:00:01.1: legacy IDE quirk: reg 0x18: [io 0x0170-0x0177] [ 0.430029] pci 0000:00:01.1: legacy IDE quirk: reg 0x1c: [io 0x0376] [ 0.433269] pci 0000:00:01.3: [8086:7113] type 00 class 0x068000 [ 0.437865] pci 0000:00:01.3: quirk: [io 0x0600-0x063f] claimed by PIIX4 ACPI [ 0.441063] pci 0000:00:01.3: quirk: [io 0x0700-0x070f] claimed by PIIX4 SMB [ 0.443904] pci 0000:00:02.0: [1af4:1000] type 00 class 0x020000 [ 0.450032] pci 0000:00:02.0: reg 0x10: [io 0xc300-0xc31f] [ 0.464035] pci 0000:00:02.0: reg 0x20: [mem 0xe0000000000-0xe0000003fff 64bit pref] [ 0.470027] pci 0000:00:02.0: reg 0x30: [mem 0xfeb80000-0xfebbffff pref] [ 0.476865] pci 0000:00:05.0: [1af4:1001] type 00 class 0x010000 [ 0.484016] pci 0000:00:05.0: reg 0x10: [io 0xc000-0xc07f] [ 0.489017] pci 0000:00:05.0: reg 0x14: [mem 0xfebc0000-0xfebc0fff] [ 0.503021] pci 0000:00:05.0: reg 0x20: [mem 0xe0000004000-0xe0000007fff 64bit pref] [ 0.513494] pci 0000:00:06.0: [1af4:1001] type 00 class 0x010000 [ 0.521022] pci 0000:00:06.0: reg 0x10: [io 0xc080-0xc0ff] [ 0.528026] pci 0000:00:06.0: reg 0x14: [mem 0xfebc1000-0xfebc1fff] [ 0.547020] pci 0000:00:06.0: reg 0x20: [mem 0xe0000008000-0xe000000bfff 64bit pref] [ 0.558732] pci 0000:00:07.0: [1af4:1001] type 00 class 0x010000 [ 0.564015] pci 0000:00:07.0: reg 0x10: [io 0xc100-0xc17f] [ 0.570021] pci 0000:00:07.0: reg 0x14: [mem 0xfebc2000-0xfebc2fff] [ 0.589017] pci 0000:00:07.0: reg 0x20: [mem 0xe000000c000-0xe000000ffff 64bit pref] [ 0.598310] pci 0000:00:08.0: [1af4:1001] type 00 class 0x010000 [ 0.609024] pci 0000:00:08.0: reg 0x10: [io 0xc180-0xc1ff] [ 0.618023] pci 0000:00:08.0: reg 0x14: [mem 0xfebc3000-0xfebc3fff] [ 0.644023] pci 0000:00:08.0: reg 0x20: [mem 0xe0000010000-0xe0000013fff 64bit pref] [ 0.657878] pci 0000:00:09.0: [1af4:1001] type 00 class 0x010000 [ 0.665025] pci 0000:00:09.0: reg 0x10: [io 0xc200-0xc27f] [ 0.672024] pci 0000:00:09.0: reg 0x14: [mem 0xfebc4000-0xfebc4fff] [ 0.689018] pci 0000:00:09.0: reg 0x20: [mem 0xe0000014000-0xe0000017fff 64bit pref] [ 0.699494] pci 0000:00:0a.0: [1af4:1001] type 00 class 0x010000 [ 0.708021] pci 0000:00:0a.0: reg 0x10: [io 0xc280-0xc2ff] [ 0.716024] pci 0000:00:0a.0: reg 0x14: [mem 0xfebc5000-0xfebc5fff] [ 0.735022] pci 0000:00:0a.0: reg 0x20: [mem 0xe0000018000-0xe000001bfff 64bit pref] [ 0.748268] ACPI: PCI: Interrupt link LNKA configured for IRQ 10 [ 0.751389] ACPI: PCI: Interrupt link LNKB configured for IRQ 10 [ 0.753373] ACPI: PCI: Interrupt link LNKC configured for IRQ 11 [ 0.756387] ACPI: PCI: Interrupt link LNKD configured for IRQ 11 [ 0.759295] ACPI: PCI: Interrupt link LNKS configured for IRQ 9 [ 0.763330] iommu: Default domain type: Passthrough [ 0.766269] SCSI subsystem initialized [ 0.768171] ACPI: bus type USB registered [ 0.769098] usbcore: registered new interface driver usbfs [ 0.771102] usbcore: registered new interface driver hub [ 0.773088] usbcore: registered new device driver usb [ 0.775194] pps_core: LinuxPPS API ver. 1 registered [ 0.777016] pps_core: Software ver. 5.3.6 - Copyright 2005-2007 Rodolfo Giometti [ 0.781080] PTP clock support registered [ 0.783213] EDAC MC: Ver: 3.0.0 [ 0.786229] PCI: Using ACPI for IRQ routing [ 0.789170] NetLabel: Initializing [ 0.791025] NetLabel: domain hash size = 128 [ 0.792015] NetLabel: protocols = UNLABELED CIPSOv4 CALIPSO [ 0.794117] NetLabel: unlabeled traffic allowed by default [ 0.797214] vgaarb: loaded [ 0.799367] hpet0: at MMIO 0xfed00000, IRQs 2, 8, 0 [ 0.801023] hpet0: 3 comparators, 64-bit 100.000000 MHz counter [ 0.807005] clocksource: Switched to clocksource kvm-clock [ 0.921840] VFS: Disk quotas dquot_6.6.0 [ 0.923541] VFS: Dquot-cache hash table entries: 512 (order 0, 4096 bytes) [ 0.926446] *** VALIDATE ramfs *** [ 0.927993] *** VALIDATE hugetlbfs *** [ 0.929897] pnp: PnP ACPI init [ 0.932680] pnp: PnP ACPI: found 6 devices [ 0.950622] clocksource: acpi_pm: mask: 0xffffff max_cycles: 0xffffff, max_idle_ns: 2085701024 ns [ 0.953992] pci_bus 0000:00: resource 4 [io 0x0000-0x0cf7 window] [ 0.956534] pci_bus 0000:00: resource 5 [io 0x0d00-0xffff window] [ 0.959321] pci_bus 0000:00: resource 6 [mem 0x000a0000-0x000bffff window] [ 0.962233] pci_bus 0000:00: resource 7 [mem 0xc0000000-0xfebfffff window] [ 0.965064] pci_bus 0000:00: resource 8 [mem 0xe0000000000-0xe007fffffff window] [ 0.967917] NET: Registered protocol family 2 [ 0.970706] IP idents hash table entries: 131072 (order: 8, 1048576 bytes, vmalloc) [ 0.975759] tcp_listen_portaddr_hash hash table entries: 4096 (order: 5, 163840 bytes, vmalloc) [ 0.980761] TCP established hash table entries: 65536 (order: 7, 524288 bytes, vmalloc) [ 0.985231] TCP bind hash table entries: 65536 (order: 9, 2097152 bytes, vmalloc) [ 0.989607] TCP: Hash tables configured (established 65536 bind 65536) [ 0.992834] MPTCP token hash table entries: 8192 (order: 6, 393216 bytes, vmalloc) [ 0.995823] UDP hash table entries: 4096 (order: 6, 393216 bytes, vmalloc) [ 1.000987] UDP-Lite hash table entries: 4096 (order: 6, 393216 bytes, vmalloc) [ 1.003920] NET: Registered protocol family 1 [ 1.006894] RPC: Registered named UNIX socket transport module. [ 1.008757] RPC: Registered udp transport module. [ 1.010541] RPC: Registered tcp transport module. [ 1.012332] RPC: Registered tcp NFSv4.1 backchannel transport module. [ 1.014011] NET: Registered protocol family 44 [ 1.014838] pci 0000:00:00.0: Limiting direct PCI/PCI transfers [ 1.016873] pci 0000:00:01.0: PIIX3: Enabling Passive Release [ 1.018863] pci 0000:00:01.0: Activating ISA DMA hang workarounds [ 1.020612] PCI: CLS 0 bytes, default 64 [ 1.021799] Unpacking initramfs... [ 2.409438] debug: unmapping init [mem 0xffff99a0bcc54000-0xffff99a0bffbffff] [ 2.413345] PCI-DMA: Using software bounce buffering for IO (SWIOTLB) [ 2.415766] software IO TLB: mapped [mem 0x00000000a8000000-0x00000000ac000000] (64MB) [ 2.418950] clocksource: tsc: mask: 0xffffffffffffffff max_cycles: 0x229835b7123, max_idle_ns: 440795242976 ns [ 2.899533] Initialise system trusted keyrings [ 2.901392] Key type blacklist registered [ 2.903809] workingset: timestamp_bits=36 max_order=20 bucket_order=0 [ 2.913306] zbud: loaded [ 2.916968] *** VALIDATE nfs *** [ 2.917934] *** VALIDATE nfs4 *** [ 2.919641] pstore: using deflate compression [ 2.923278] Platform Keyring initialized [ 3.031490] NET: Registered protocol family 38 [ 3.033177] Key type asymmetric registered [ 3.034731] Asymmetric key parser 'x509' registered [ 3.036255] Block layer SCSI generic (bsg) driver version 0.4 loaded (major 247) [ 3.039667] io scheduler mq-deadline registered [ 3.041477] io scheduler kyber registered [ 3.042726] io scheduler bfq registered [ 3.044611] atomic64_test: passed for x86-64 platform with CX8 and with SSE [ 3.048238] shpchp: Standard Hot Plug PCI Controller Driver version: 0.4 [ 3.053902] input: Power Button as /devices/LNXSYSTM:00/LNXPWRBN:00/input/input0 [ 3.056802] ACPI: Power Button [PWRF] [ 3.062181] ACPI: \_SB_.LNKB: Enabled at IRQ 10 [ 3.067939] ACPI: \_SB_.LNKA: Enabled at IRQ 11 [ 3.086558] ACPI: \_SB_.LNKC: Enabled at IRQ 11 [ 3.096385] ACPI: \_SB_.LNKD: Enabled at IRQ 10 [ 3.118090] Serial: 8250/16550 driver, 4 ports, IRQ sharing enabled [ 3.158176] 00:03: ttyS1 at I/O 0x2f8 (irq = 3, base_baud = 115200) is a 16550A [ 3.202855] 00:04: ttyS0 at I/O 0x3f8 (irq = 4, base_baud = 115200) is a 16550A [ 3.208715] Non-volatile memory driver v1.3 [ 3.210665] Linux agpgart interface v0.103 [ 3.252118] virtio_blk virtio1: [vda] 149768 512-byte logical blocks (76.7 MB/73.1 MiB) [ 3.254815] vda: detected capacity change from 0 to 76681216 [ 3.273426] virtio_blk virtio2: [vdb] 2097152 512-byte logical blocks (1.07 GB/1.00 GiB) [ 3.276578] vdb: detected capacity change from 0 to 1073741824 [ 3.290898] virtio_blk virtio3: [vdc] 5120000 512-byte logical blocks (2.62 GB/2.44 GiB) [ 3.292904] vdc: detected capacity change from 0 to 2621440000 [ 3.307165] virtio_blk virtio4: [vdd] 5120000 512-byte logical blocks (2.62 GB/2.44 GiB) [ 3.310360] vdd: detected capacity change from 0 to 2621440000 [ 3.325222] virtio_blk virtio5: [vde] 8388608 512-byte logical blocks (4.29 GB/4.00 GiB) [ 3.328028] vde: detected capacity change from 0 to 4294967296 [ 3.341403] virtio_blk virtio6: [vdf] 8388608 512-byte logical blocks (4.29 GB/4.00 GiB) [ 3.344038] vdf: detected capacity change from 0 to 4294967296 [ 3.351757] libphy: Fixed MDIO Bus: probed [ 3.359989] usbcore: registered new interface driver usbserial_generic [ 3.362352] usbserial: USB Serial support registered for generic [ 3.364540] i8042: PNP: PS/2 Controller [PNP0303:KBD,PNP0f13:MOU] at 0x60,0x64 irq 1,12 [ 3.368124] serio: i8042 KBD port at 0x60,0x64 irq 1 [ 3.369907] serio: i8042 AUX port at 0x60,0x64 irq 12 [ 3.372509] mousedev: PS/2 mouse device common for all mice [ 3.375541] rtc_cmos 00:05: RTC can wake from S4 [ 3.378306] input: AT Translated Set 2 keyboard as /devices/platform/i8042/serio0/input/input1 [ 3.378573] rtc_cmos 00:05: registered as rtc0 [ 3.382959] rtc_cmos 00:05: alarms up to one day, y3k, 242 bytes nvram, hpet irqs [ 3.385129] input: VirtualPS/2 VMware VMMouse as /devices/platform/i8042/serio1/input/input4 [ 3.388607] intel_pstate: CPU model not supported [ 3.394164] input: VirtualPS/2 VMware VMMouse as /devices/platform/i8042/serio1/input/input3 [ 3.395804] hid: raw HID events driver (C) Jiri Kosina [ 3.399991] usbcore: registered new interface driver usbhid [ 3.401621] usbhid: USB HID core driver [ 3.403290] drop_monitor: Initializing network drop monitor service [ 3.405933] Initializing XFRM netlink socket [ 3.407953] NET: Registered protocol family 10 [ 3.411192] Segment Routing with IPv6 [ 3.412883] NET: Registered protocol family 17 [ 3.415641] mpls_gso: MPLS GSO support [ 3.421534] RAS: Correctable Errors collector initialized. [ 3.423763] AVX version of gcm_enc/dec engaged. [ 3.425425] AES CTR mode by8 optimization enabled [ 3.497994] sched_clock: Marking stable (3497961028, 0)->(4428056654, -930095626) [ 3.502310] registered taskstats version 1 [ 3.504531] Loading compiled-in X.509 certificates [ 3.506916] zswap: loaded using pool lzo/zbud [ 3.527670] Key type big_key registered [ 3.540651] Key type encrypted registered [ 3.542774] ima: No TPM chip found, activating TPM-bypass! [ 3.545969] ima: Allocated hash algorithm: sha1 [ 3.548271] ima: No architecture policies found [ 3.550658] evm: Initialising EVM extended attributes: [ 3.553307] evm: security.selinux [ 3.555199] evm: security.ima [ 3.556467] evm: security.capability [ 3.557716] evm: HMAC attrs: 0x1 [ 3.560078] rtc_cmos 00:05: setting system clock to 2026-09-05 20:05:00 UTC (1788638700) [ 3.567528] debug: unmapping init [mem 0xffffffff88203000-0xffffffff883fffff] [ 3.570775] debug: unmapping init [mem 0xffffffff86f82000-0xffffffff87258fff] [ 3.579092] Write protecting the kernel read-only data: 28672k [ 3.582578] debug: unmapping init [mem 0xffffffff85603000-0xffffffff857fffff] [ 3.585483] debug: unmapping init [mem 0xffffffff85f14000-0xffffffff85ffffff] [ 3.621751] systemd[1]: systemd 239 (239-82.el8_10.5) running in system mode. (+PAM +AUDIT +SELINUX +IMA -APPARMOR +SMACK +SYSVINIT +UTMP +LIBCRYPTSETUP +GCRYPT +GNUTLS +ACL +XZ +LZ4 +SECCOMP +BLKID +ELFUTILS +KMOD +IDN2 -IDN +PCRE2 default-hierarchy=legacy) [ 3.632752] systemd[1]: Detected virtualization kvm. [ 3.635094] systemd[1]: Detected architecture x86-64. [ 3.637383] systemd[1]: Running in initial RAM disk. Welcome to Rocky Linux 8.10 (Green Obsidian) dracut-049-233.git20240115.el8 (Initramfs)! [ 3.667220] systemd[1]: No hostname configured. [ 3.669129] systemd[1]: Set hostname to . [ 3.670771] random: systemd: uninitialized urandom read (16 bytes read) [ 3.673067] systemd[1]: Initializing machine ID from random generator. [ 3.716324] random: ln: uninitialized urandom read (6 bytes read) [ 3.820252] random: systemd: uninitialized urandom read (16 bytes read) [ 3.823435] systemd[1]: Listening on udev Control Socket. [ OK ] Listening on udev Control Socket. [ 3.827709] systemd[1]: Reached target Slices. [ OK ] Reached target Slices. [ 3.834108] systemd[1]: Listening on Journal Socket (/dev/log). [ OK ] Listening on Journal Socket (/dev/log). [ OK ] Listening on udev Kernel Socket. [ OK ] Reached target Local File Systems. [ OK ] Reached target Timers. [ OK ] Started Dispatch Password Requests to Console Directory Watch. [ OK ] Reached target Local Encrypted Volumes. [ OK ] Reached target Initrd Root Device. [ OK ] Listening on Journal Socket. Starting Create Volatile Files and Directories... [ OK ] Reached target Sockets. Starting Create list of required st…ce nodes for the current kernel... [ OK ] Started Memstrack Anylazing Service. Starting Apply Kernel Variables... Starting Journal Service... [ OK ] Reached target Swap. [ OK ] Reached target Paths. Starting Setup Virtual Console... [ OK ] Started Create Volatile Files and Directories. [ OK ] Started Create list of required sta…vice nodes for the current kernel. [ OK ] Started Apply Kernel Variables. [ OK ] Started Setup Virtual Console. Starting dracut cmdline hook... Starting Create Static Device Nodes in /dev... [ OK ] Started Create Static Device Nodes in /dev. [ OK ] Started Journal Service. [ OK ] Started dracut cmdline hook. Starting dracut pre-udev hook... [ 4.467126] device-mapper: uevent: version 1.0.3 [ 4.469213] device-mapper: ioctl: 4.46.0-ioctl (2022-02-22) initialised: dm-devel@redhat.com [ OK ] Started dracut pre-udev hook. Starting udev Kernel Device Manager... [ OK ] Started udev Kernel Device Manager. Starting dracut pre-trigger hook... [ OK ] Started dracut pre-trigger hook. Starting udev Coldplug all Devices... Mounting Kernel Configuration File System... [ OK ] Mounted Kernel Configuration File System. [ OK ] Started udev Coldplug all Devices. [ OK ] Reached target System Initialization. [ OK ] Reached target Basic System. [ OK ] Started Hardware RNG Entropy Gatherer Daemon. Starting dracut initqueue hook... [ 5.290790] random: fast init done [ 5.307237] virtio_net virtio0 ens2: renamed from eth0 [ 5.400292] scsi host0: ata_piix [ 5.419703] scsi host1: ata_piix [ 5.425806] ata1: PATA max MWDMA2 cmd 0x1f0 ctl 0x3f6 bmdma 0xc320 irq 14 [ 5.428344] ata2: PATA max MWDMA2 cmd 0x170 ctl 0x376 bmdma 0xc328 irq 15 [ 9.104878] dracut-initqueue[591]: RTNETLINK answers: File exists [ 10.069470] random: crng init done [ 10.070671] random: 7 urandom warning(s) missed due to ratelimiting Starting nbd nbd0... [ OK ] Started nbd nbd0. [ OK ] Started dracut initqueue hook. [ OK ] Reached target Remote File Systems (Pre). [ OK ] Reached target Remote File Systems. Mounting /sysroot... [ 10.521249] EXT4-fs (nbd0): mounted filesystem with ordered data mode. Opts: (null) [ OK ] Mounted /sysroot. [ OK ] Reached target Initrd Root File System. Starting Reload Configuration from the Real Root... [ OK ] Started Reload Configuration from the Real Root. [ OK ] Reached target Initrd File Systems. [ OK ] Reached target Initrd Default Target. Starting dracut pre-pivot and cleanup hook... [ OK ] Started dracut pre-pivot and cleanup hook. Starting Cleaning Up and Shutting Down Daemons... [ OK ] Stopped dracut pre-pivot and cleanup hook. [ OK ] Stopped target Remote File Systems. [ OK ] Stopped target Timers. [ OK ] Stopped target Initrd Default Target. [ OK ] Stopped target Initrd Root Device. Stopping Hardware RNG Entropy Gatherer Daemon... [ OK ] Stopped target Remote File Systems (Pre). [ OK ] Stopped dracut initqueue hook. [ OK ] Stopped Hardware RNG Entropy Gatherer Daemon. [ OK ] Stopped target Basic System. [ OK ] Stopped target System Initialization. [ OK ] Stopped Apply Kernel Variables. [ OK ] Stopped Create Volatile Files and Directories. [ OK ] Stopped target Local Encrypted Volumes. [ OK ] Stopped target Local File Systems. [ OK ] Stopped udev Coldplug all Devices. [ OK ] Stopped dracut pre-trigger hook. Stopping udev Kernel Device Manager... [ OK ] Stopped target Swap. [ OK ] Stopped target Slices. [ OK ] Stopped target Paths. [ OK ] Stopped Dispatch Password Requests to Console Directory Watch. [ OK ] Stopped target Sockets. [ OK ] Started Cleaning Up and Shutting Down Daemons. [ OK ] Stopped udev Kernel Device Manager. [ OK ] Stopped dracut pre-udev hook. [ OK ] Stopped dracut cmdline hook. [ OK ] Stopped Create Static Device Nodes in /dev. [ OK ] Stopped Create list of required sta…vice nodes for the current kernel. [ OK ] Closed udev Control Socket. [ OK ] Closed udev Kernel Socket. Starting Cleanup udevd DB... [ OK ] Started Cleanup udevd DB. [ OK ] Reached target Switch Root. Starting Switch Root... [ 11.622150] printk: systemd: 26 output lines suppressed due to ratelimiting [ 11.890968] SELinux: Disabled at runtime. [ 11.956529] systemd[1]: systemd 239 (239-82.el8_10.5) running in system mode. (+PAM +AUDIT +SELINUX +IMA -APPARMOR +SMACK +SYSVINIT +UTMP +LIBCRYPTSETUP +GCRYPT +GNUTLS +ACL +XZ +LZ4 +SECCOMP +BLKID +ELFUTILS +KMOD +IDN2 -IDN +PCRE2 default-hierarchy=legacy) [ 11.963471] systemd[1]: Detected virtualization kvm. [ 11.964994] systemd[1]: Detected architecture x86-64. Welcome to Rocky Linux 8.10 (Green Obsidian)! [ 12.458852] systemd[1]: initrd-switch-root.service: Succeeded. [ 12.461972] systemd[1]: Stopped Switch Root. [ OK ] Stopped Switch Root. [ 12.465459] systemd[1]: systemd-journald.service: Service has no hold-off time (RestartSec=0), scheduling restart. [ 12.469388] systemd[1]: systemd-journald.service: Scheduled restart job, restart counter is at 1. [ 12.471850] systemd[1]: Stopped Journal Service. [ OK ] Stopped Journal Service. [ 12.483102] systemd[1]: Starting Journal Service... Starting Journal Service... [ 12.490908] systemd[1]: Created slice system-serial\x2dgetty.slice. [ OK ] Created slice system-serial\x2dgetty.slice. [ OK ] Created slice system-getty.slice. [ OK ] Started Dispatch Password Requests to Console Directory Watch. Starting Remount Root and Kernel File Systems... [ OK ] Created slice system-sshd\x2dkeygen.slice. Mounting POSIX Message Queue File System... [ OK ] Reached target rpc_pipefs.target. [ OK ] Stopped target Switch Root. Starting Apply Kernel Variables... Starting Create list of required st…ce nodes for the current kernel... [FAILED] Failed to set up automount Arbitrar…rmats File System Automount Point. See 'systemctl status proc-sys-fs-binfmt_misc.automount' for details. [ OK ] Listening on udev Control Socket. Mounting Kernel Debug File System... [ OK ] Stopped target Initrd File Systems. [ OK ] Stopped target Initrd Root File System. [ OK ] Listening on initctl Compatibility Named Pipe. [ OK ] Listening on Process Core Dump Socket. [ OK ] Listening on udev Kernel Socket. Starting udev Coldplug all Devices... Mounting Huge Pages File System... [ OK ] Started Forward Password Requests to Wall Directory Watch. [ OK ] Reached target Paths. [ OK ] Reached target Local Encrypted Volumes. [ OK ] Listening on RPCbind Server Activation Socket. [ OK ] Reached target RPC Port Mapper. Activating swap /dev/disk/by-label/SWAP... [ OK ] Created slice User and Session Slice. [ OK ] Reached target Slices. [ 12.659792] Adding 1048572k swap on /dev/vdb. Priority:-2 extents:1 across:1048572k FS [ OK ] Started Journal Service. [FAILED] Failed to start Remount Root and Kernel File Systems. See 'systemctl status systemd-remount-fs.service' for details. [ OK ] Mounted POSIX Message Queue File System. [ OK ] Started Apply Kernel Variables. [ OK ] Started Create list of required sta…vice nodes for the current kernel. [ OK ] Mounted Kernel Debug File System. [ OK ] Mounted Huge Pages File System. [ OK ] Activated swap /dev/disk/by-label/SWAP. [ OK ] Reached target Swap. Starting Configure read-only root support... Starting Create Static Device Nodes in /dev... Starting Flush Journal to Persistent Storage... [ OK ] Started Flush Journal to Persistent Storage. [ OK ] Started Create Static Device Nodes in /dev. [ OK ] Reached target Local File Systems (Pre). Mounting /home/green/git/lustre-release... Mounting /mnt... Starting udev Kernel Device Manager... [ OK ] Mounted /mnt. [ OK ] Started udev Coldplug all Devices. [ 12.901946] squashfs: version 4.0 (2009/01/31) Phillip Lougher [ OK ] Mounted /home/green/git/lustre-release. [ OK ] Started udev Kernel Device Manager. [ 13.206687] input: PC Speaker as /devices/platform/pcspkr/input/input5 [ 13.290895] piix4_smbus 0000:00:01.3: SMBus Host Controller at 0x700, revision 0 [ 13.373874] RAPL PMU: API unit is 2^-32 Joules, 0 fixed counters, 10737418240 ms ovfl timer [ 13.394741] EDAC sbridge: Ver: 1.1.2 [ 14.619204] Key type dns_resolver registered [ 14.923724] NFS: Registering the id_resolver key type [ 14.925615] Key type id_resolver registered [ 14.926891] Key type id_legacy registered [ OK ] Started Configure read-only root support. [ OK ] Reached target Local File Systems. Starting Mark the need to relabel after reboot... Starting Rebuild Dynamic Linker Cache... Starting Load/Save Random Seed... Starting Create Volatile Files and Directories... [ OK ] Started Mark the need to relabel after reboot. [ OK ] Started Load/Save Random Seed. [ OK ] Started Create Volatile Files and Directories. Starting RPC Bind... Starting Update UTMP about System Boot/Shutdown... [ OK ] Started Update UTMP about System Boot/Shutdown. [ OK ] Started RPC Bind. [ OK ] Started Rebuild Dynamic Linker Cache. Starting Update is Completed... [ OK ] Started Update is Completed. [ OK ] Reached target System Initialization. [ OK ] Started Daily Cleanup of Temporary Directories. [ OK ] Started daily update of the root trust anchor for DNSSEC. [ OK ] Started dnf makecache --timer. [ OK ] Reached target Timers. [ OK ] Listening on D-Bus System Message Bus Socket. [ OK ] Reached target Sockets. [ OK ] Reached target Basic System. [ OK ] Started D-Bus System Message Bus. [ OK ] Reached target sshd-keygen.target. Starting Network Manager... Starting Login Service... Starting Restore /run/initramfs on shutdown... [ OK ] Started Hardware RNG Entropy Gatherer Daemon. [ OK ] Started irqbalance daemon. [ OK ] Started Restore /run/initramfs on shutdown. [ OK ] Started Login Service. [ OK ] Started Network Manager. Starting Network Manager Wait Online... [ OK ] Reached target Network. Starting Dynamic System Tuning Daemon... Starting OpenSSH server daemon... Starting GSSAPI Proxy Daemon... Starting Hostname Service... [ OK ] Started OpenSSH server daemon. [ OK ] Started GSSAPI Proxy Daemon. [ OK ] Reached target NFS client services. [ OK ] Reached target Remote File Systems (Pre). [ OK ] Reached target Remote File Systems. Starting Permit User Sessions... [ OK ] Started Permit User Sessions. [ OK ] Started Command Scheduler. [ OK ] Started Serial Getty on ttyS1. [ OK ] Started Getty on tty1. [ OK ] Started Serial Getty on ttyS0. [ OK ] Reached target Login Prompts. [ OK ] Started Hostname Service. Starting Network Manager Script Dispatcher Service... [ OK ] Started Network Manager Script Dispatcher Service. [ OK ] Started Network Manager Wait Online. [ OK ] Reached target Network is Online. Starting System Logging Service... Starting Notify NFS peers of a restart... Starting Crash recovery kernel arming... [ OK ] Started Notify NFS peers of a restart. [ OK ] Started System Logging Service. Starting Authorization Manager... [ OK ] Started Authorization Manager. [ OK ] Started Dynamic System Tuning Daemon. [ OK ] Reached target Multi-User System. [ OK ] Reached target Graphical Interface. Starting Update UTMP about System Runlevel Changes... [ OK ] Started Update UTMP about System Runlevel Changes. Rocky Linux 8.10 (Green Obsidian) Kernel 4.18.0rh8.10-debug on an x86_64 oleg350-server login: [ 64.138055] hrtimer: interrupt took 5401079 ns [ 65.080236] libcfs: loading out-of-tree module taints kernel. [ 65.194410] Key type ._llcrypt registered [ 65.204574] Key type .llcrypt registered [ 65.423116] Lustre: DEBUG MARKER: oleg350-server.virtnet: executing set_hostid [ 84.549474] Lustre: DEBUG MARKER: oleg350-server.virtnet: executing load_modules_local [ 86.211450] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 86.234630] alg: No test for adler32 (adler32-zlib) [ 87.725394] Lustre: Lustre: Build Version: 2.17.58_3_g980e1e2 [ 88.632562] LNet: Added LNI 192.168.203.150@tcp [8/256/0/180] [ 90.455173] Key type lgssc registered [ 92.264109] Lustre: Echo OBD driver; http://www.lustre.org/ [ 111.338443] ZFS: Loaded module v2.3.2-1, ZFS pool version 5000, ZFS filesystem version 5 [ 153.762670] Lustre: DEBUG MARKER: oleg350-server.virtnet: executing load_modules_local [ 167.572058] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 167.617738] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 168.919438] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 168.982497] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 169.172350] Lustre: lustre-MDT0000: new disk, initializing [ 169.298677] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 169.325180] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 174.660415] Lustre: DEBUG MARKER: oleg350-server.virtnet: executing set_default_debug vfstrace rpctrace dlmtrace neterror ha config ioctl super lfsck all [ 188.838212] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 188.951971] Lustre: 6514:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 188.983269] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [ 188.990646] Lustre: Skipped 1 previous similar message [ 189.117517] Lustre: lustre-MDT0001: new disk, initializing [ 189.236935] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 189.286944] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [ 189.338513] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 193.807423] Lustre: DEBUG MARKER: oleg350-server.virtnet: executing set_default_debug vfstrace rpctrace dlmtrace neterror ha config ioctl super lfsck all [ 199.564827] Lustre: Modifying parameter general.debug_raw_pointers=Y in log params [ 210.504954] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 210.754335] Lustre: lustre-OST0000: new disk, initializing [ 210.759927] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 210.767992] Lustre: 8452:0:(osd_compat.c:1353:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 210.876201] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 218.394813] Lustre: DEBUG MARKER: oleg350-server.virtnet: executing set_default_debug vfstrace rpctrace dlmtrace neterror ha config ioctl super lfsck all [ 220.701811] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [ 220.708966] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 220.825237] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 234.155892] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 234.347880] Lustre: lustre-OST0001: new disk, initializing [ 234.353345] Lustre: srv-lustre-OST0001: No data found on store. Initialize space. [ 234.364452] Lustre: 9527:0:(osd_compat.c:1353:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 234.456632] Lustre: lustre-OST0001: Imperative Recovery not enabled, recovery window 60-180 [ 240.400879] Lustre: DEBUG MARKER: oleg350-server.virtnet: executing set_default_debug vfstrace rpctrace dlmtrace neterror ha config ioctl super lfsck all [ 242.191394] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x00000002c0000400-0x0000000300000400]:1:ost [ 242.200131] Lustre: cli-lustre-OST0001-super: Allocated super-sequence [0x00000002c0000400-0x0000000300000400]:1:ost] [ 242.298772] Lustre: lustre-OST0001-osc-MDT0000: update sequence from 0x100010000 to 0x2c0000401 [ 253.760673] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 261.290418] Lustre: Setting parameter general.lod.*.mdt_hash=crush in log params [ 268.858786] Lustre: DEBUG MARKER: oleg350-server.virtnet: executing check_logdir /tmp/testlogs/ [ 275.162405] Lustre: DEBUG MARKER: oleg350-server.virtnet: executing yml_node [ 280.786651] Lustre: DEBUG MARKER: Client: 2.17.58.3 [ 283.879787] Lustre: DEBUG MARKER: MDS: 2.17.58.3 [ 286.723343] Lustre: DEBUG MARKER: OSS: 2.17.58.3 [ 288.319165] Lustre: DEBUG MARKER: -----============= acceptance-small: conf-sanity ============----- Sat Sep 5 16:09:44 EDT 2026 [ 306.822691] Lustre: DEBUG MARKER: excepting tests: 102 106 115 32newtarball [ 308.692489] Lustre: DEBUG MARKER: skipping tests SLOW=no: 45 69 106 111 114 [ 327.657361] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 327.672725] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 327.697986] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 329.183671] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 329.185241] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 329.198584] Lustre: Skipped 1 previous similar message [ 329.225972] Lustre: Skipped 1 previous similar message [ 331.039075] Lustre: server umount lustre-MDT0000 complete [ 335.024572] LustreError: 9526:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1788639031 with bad export cookie 16857562073661309827 [ 335.031606] LustreError: MGC192.168.203.150@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 335.032500] LustreError: 9526:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 339.423750] LustreError: 6520:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 339.423940] Lustre: lustre-MDT0001-lwp-OST0001: Connection to lustre-MDT0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 339.425070] Lustre: lustre-MDT0001: Not available for connect from 0@lo (stopping) [ 339.425076] Lustre: Skipped 2 previous similar messages [ 339.447942] LustreError: 6520:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 6 previous similar messages [ 339.480212] Lustre: Skipped 2 previous similar messages [ 340.447995] LustreError: 6521:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 340.465697] LustreError: 6521:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 1 previous similar message [ 341.579706] Lustre: server umount lustre-MDT0001 complete [ 346.259751] Lustre: server umount lustre-OST0000 complete [ 350.336499] Lustre: server umount lustre-OST0001 complete [ 357.842603] Lustre: DEBUG MARKER: oleg350-server.virtnet: executing set_hostid [ 368.279571] Lustre: DEBUG MARKER: oleg350-server.virtnet: executing load_modules_local [ 407.942980] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 408.281239] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 408.335158] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 408.407706] Lustre: lustre-MDT0000: new disk, initializing [ 408.487967] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 408.506519] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 413.358481] Lustre: DEBUG MARKER: oleg350-server.virtnet: executing set_default_debug -1 all [ 423.665688] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 423.832491] Lustre: 15083:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 423.909516] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [ 423.914941] Lustre: Skipped 1 previous similar message [ 423.991740] Lustre: lustre-MDT0001: new disk, initializing [ 424.138953] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 424.209122] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [ 424.227347] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 430.158910] Lustre: DEBUG MARKER: oleg350-server.virtnet: executing set_default_debug -1 all [ 441.325787] Lustre: DEBUG MARKER: oleg350-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 448.342710] Lustre: DEBUG MARKER: oleg350-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 456.043107] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 456.366929] Lustre: lustre-OST0000: new disk, initializing [ 456.370812] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 456.382141] Lustre: 17049:0:(osd_compat.c:1353:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 456.515472] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 457.660218] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [ 457.671464] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 457.815986] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 462.902337] Lustre: DEBUG MARKER: oleg350-server.virtnet: executing set_default_debug -1 all [ 472.392660] Lustre: DEBUG MARKER: oleg350-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 476.599280] Lustre: DEBUG MARKER: oleg350-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 [ 476.870901] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [ 480.945613] Lustre: DEBUG MARKER: oleg350-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid 50 [ 481.144601] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [ 483.297728] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 483.306085] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 483.308743] LustreError: Skipped 1 previous similar message [ 483.314117] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 483.317519] Lustre: Skipped 1 previous similar message [ 483.333290] Lustre: Skipped 4 previous similar messages [ 488.422690] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 488.432691] Lustre: Skipped 1 previous similar message [ 489.375169] Lustre: server umount lustre-OST0000 complete [ 493.543082] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 493.571231] Lustre: Skipped 1 previous similar message [ 498.656610] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 498.671053] Lustre: Skipped 3 previous similar messages [ 498.973463] Lustre: server umount lustre-MDT0000 complete [ 502.270594] LustreError: 15072:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1788639199 with bad export cookie 16857562073661311360 [ 502.278196] LustreError: MGC192.168.203.150@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 502.288144] LustreError: 15072:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 502.516747] Lustre: server umount lustre-MDT0001 complete [ 508.080363] Lustre: DEBUG MARKER: == conf-sanity test 76a: set permanent params with lctl across mounts ========================================================== 16:13:23 (1788639203) [ 515.024327] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 515.590195] LustreError: 19316:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 515.672143] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 520.678249] LustreError: 19317:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 520.822185] Lustre: DEBUG MARKER: oleg350-server.virtnet: executing set_default_debug -1 all [ 525.804224] LustreError: 19316:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 530.373131] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 536.376837] Lustre: DEBUG MARKER: oleg350-server.virtnet: executing set_default_debug -1 all [ 545.230061] Lustre: DEBUG MARKER: oleg350-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 551.439509] Lustre: DEBUG MARKER: oleg350-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 558.404471] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 565.459784] Lustre: DEBUG MARKER: oleg350-server.virtnet: executing set_default_debug -1 all [ 573.742888] Lustre: DEBUG MARKER: oleg350-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 580.956507] Lustre: Modifying parameter general.osc.*.max_dirty_mb=452 in log params [ 587.235681] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 587.243877] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 587.268305] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 593.031328] Lustre: server umount lustre-MDT0000 complete [ 595.943072] LustreError: 19317:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 595.976219] LustreError: 19317:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 2 previous similar messages [ 597.667337] LustreError: 19298:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1788639294 with bad export cookie 16857562073661312242 [ 597.674033] LustreError: MGC192.168.203.150@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 597.685475] LustreError: 19298:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 598.216925] Lustre: server umount lustre-MDT0001 complete [ 613.076672] Lustre: server umount lustre-OST0000 complete [ 638.199347] Lustre: DEBUG MARKER: oleg350-server.virtnet: executing load_modules_local [ 649.501115] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 650.131371] LustreError: 23700:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 650.220743] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 650.224052] Lustre: Skipped 2 previous similar messages [ 655.651730] Lustre: DEBUG MARKER: oleg350-server.virtnet: executing set_default_debug -1 all [ 665.148562] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 669.904806] Lustre: DEBUG MARKER: oleg350-server.virtnet: executing set_default_debug -1 all [ 673.241633] Lustre: Setting parameter general.debug_raw_pointers=Y in log params [ 680.591966] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 687.971355] Lustre: DEBUG MARKER: oleg350-server.virtnet: executing set_default_debug -1 all [ 698.698745] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 698.956756] Lustre: lustre-OST0001: new disk, initializing [ 698.963581] Lustre: srv-lustre-OST0001: No data found on store. Initialize space. [ 698.977309] Lustre: 26015:0:(osd_compat.c:1353:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 700.560036] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x00000002c0000400-0x0000000300000400]:1:ost [ 700.581987] Lustre: cli-lustre-OST0001-super: Allocated super-sequence [0x00000002c0000400-0x0000000300000400]:1:ost] [ 700.736788] Lustre: lustre-OST0001-osc-MDT0000: update sequence from 0x100010000 to 0x2c0000401 [ 707.057738] Lustre: DEBUG MARKER: oleg350-server.virtnet: executing set_default_debug -1 all [ 718.833944] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 723.063299] Lustre: Setting parameter general.lod.*.mdt_hash=crush in log params [ 741.868704] Lustre: lustre-MDT0000-lwp-OST0000: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 741.888297] Lustre: Skipped 3 previous similar messages [ 741.892918] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 741.914826] Lustre: Skipped 6 previous similar messages [ 744.295636] Lustre: server umount lustre-MDT0000 complete [ 746.980539] LustreError: 24432:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 747.016900] LustreError: 24432:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 5 previous similar messages [ 748.563282] LustreError: 26783:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1788639445 with bad export cookie 16857562073661313089 [ 748.570873] LustreError: MGC192.168.203.150@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 748.577396] LustreError: 26783:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 3 previous similar messages [ 749.084384] Lustre: server umount lustre-MDT0001 complete [ 763.664295] Lustre: server umount lustre-OST0000 complete [ 777.410718] Lustre: server umount lustre-OST0001 complete [ 794.932503] Lustre: DEBUG MARKER: oleg350-server.virtnet: executing load_modules_local [ 804.742826] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 805.341183] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 805.349353] Lustre: Skipped 3 previous similar messages [ 809.501155] Lustre: DEBUG MARKER: oleg350-server.virtnet: executing set_default_debug -1 all [ 815.587942] LustreError: 29047:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 815.606153] LustreError: 29047:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 2 previous similar messages [ 817.315369] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 821.726628] Lustre: DEBUG MARKER: oleg350-server.virtnet: executing set_default_debug -1 all [ 824.264486] Lustre: 30193:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify general/debug_raw_pointers=Y (mode = 0) failed: rc = -17 [ 832.285547] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 841.061864] Lustre: DEBUG MARKER: oleg350-server.virtnet: executing set_default_debug -1 all [ 850.656445] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 857.980976] Lustre: DEBUG MARKER: oleg350-server.virtnet: executing set_default_debug -1 all [ 866.687859] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 870.473395] Lustre: 32071:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify general/lod.*.mdt_hash=crush (mode = 0) failed: rc = -17 [ 879.072157] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 879.079355] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 879.104889] Lustre: Skipped 1 previous similar message [ 879.112415] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 883.360509] Lustre: server umount lustre-MDT0000 complete [ 886.979211] LustreError: 29797:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1788639583 with bad export cookie 16857562073661314384 [ 886.984922] LustreError: MGC192.168.203.150@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 886.986845] LustreError: 29797:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 1 previous similar message [ 887.307255] Lustre: server umount lustre-MDT0001 complete [ 901.239995] Lustre: server umount lustre-OST0000 complete [ 914.413658] Lustre: server umount lustre-OST0001 complete [ 924.025959] Lustre: DEBUG MARKER: == conf-sanity test 76b: verify params log setup correctly ========================================================== 16:20:19 (1788639619) [ 954.586374] Lustre: DEBUG MARKER: oleg350-server.virtnet: executing load_modules_local [ 966.779460] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 967.217229] LustreError: 34878:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 967.232677] LustreError: 34878:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 8 previous similar messages [ 972.456882] Lustre: DEBUG MARKER: oleg350-server.virtnet: executing set_default_debug -1 all [ 981.903335] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 987.433619] Lustre: DEBUG MARKER: oleg350-server.virtnet: executing set_default_debug -1 all [ 990.515300] Lustre: 36028:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify general/debug_raw_pointers=Y (mode = 0) failed: rc = -17 [ 998.002896] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 1005.973500] Lustre: DEBUG MARKER: oleg350-server.virtnet: executing set_default_debug -1 all [ 1015.329525] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 1023.173968] Lustre: DEBUG MARKER: oleg350-server.virtnet: executing set_default_debug -1 all [ 1032.863909] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 1037.823692] Lustre: 37906:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify general/lod.*.mdt_hash=crush (mode = 0) failed: rc = -17 [ 1043.946149] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 1043.955945] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 1043.974358] Lustre: Skipped 3 previous similar messages [ 1043.982808] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 1043.985183] Lustre: Skipped 4 previous similar messages [ 1049.974832] Lustre: server umount lustre-MDT0000 complete [ 1054.386438] LustreError: 37908:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1788639751 with bad export cookie 16857562073661315490 [ 1054.400166] LustreError: MGC192.168.203.150@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 1054.406702] LustreError: 37908:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 1054.827509] Lustre: server umount lustre-MDT0001 complete [ 1068.332502] Lustre: server umount lustre-OST0000 complete [ 1083.052481] Lustre: server umount lustre-OST0001 complete [ 1093.604963] Lustre: DEBUG MARKER: == conf-sanity test 76c: verify changelog_mask is applied with lctl set_param -P ========================================================== 16:23:08 (1788639788) [ 1112.754225] Lustre: DEBUG MARKER: oleg350-server.virtnet: executing load_modules_local [ 1123.910282] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1124.293476] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 1124.302176] Lustre: Skipped 7 previous similar messages [ 1129.269321] Lustre: DEBUG MARKER: oleg350-server.virtnet: executing set_default_debug -1 all [ 1137.626592] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1143.462374] Lustre: DEBUG MARKER: oleg350-server.virtnet: executing set_default_debug -1 all [ 1146.618453] Lustre: 41196:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify general/debug_raw_pointers=Y (mode = 0) failed: rc = -17 [ 1153.733269] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 1160.936588] Lustre: DEBUG MARKER: oleg350-server.virtnet: executing set_default_debug -1 all [ 1170.040897] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 1175.878549] Lustre: DEBUG MARKER: oleg350-server.virtnet: executing set_default_debug -1 all [ 1184.910167] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 1189.485233] Lustre: 43071:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify general/lod.*.mdt_hash=crush (mode = 0) failed: rc = -17 [ 1190.380332] Lustre: Setting parameter general.mdd.*.changelog_mask=-CLOSE in log params [ 1190.396562] Lustre: Skipped 1 previous similar message [ 1194.465058] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 1194.481639] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 1194.498664] Lustre: Skipped 3 previous similar messages [ 1196.368854] Lustre: server umount lustre-MDT0000 complete [ 1201.346234] LustreError: 40030:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1788639898 with bad export cookie 16857562073661316596 [ 1201.357794] LustreError: MGC192.168.203.150@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 1201.369099] LustreError: 40030:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 1207.891000] Lustre: server umount lustre-MDT0001 complete [ 1218.594925] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1226.737112] Lustre: Evicted from MGS (at 0@lo) after server handle changed from 0x0 to 0xe9f217640bae223f [ 1226.760138] Lustre: MGC192.168.203.150@tcp: Connection restored to 0@lo (at 0@lo) [ 1227.284177] LustreError: 41559:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 1227.312731] LustreError: 41559:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 24 previous similar messages [ 1232.927278] Lustre: DEBUG MARKER: oleg350-server.virtnet: executing set_default_debug -1 all [ 1236.653180] Lustre: lustre-MDT0000-lwp-OST0000: Connection restored to 0@lo (at 0@lo) [ 1242.394696] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1246.963936] Lustre: lustre-MDT0001-lwp-OST0000: Connection restored to 0@lo (at 0@lo) [ 1246.973745] Lustre: Skipped 1 previous similar message [ 1247.411948] Lustre: DEBUG MARKER: oleg350-server.virtnet: executing set_default_debug -1 all [ 1255.268620] Lustre: DEBUG MARKER: oleg350-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 1257.419570] Lustre: DEBUG MARKER: mdc.lustre-MDT0000-mdc-*.mds_server_uuid in FULL state after 0 sec [ 1263.864851] Lustre: DEBUG MARKER: oleg350-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 1265.887882] Lustre: DEBUG MARKER: mdc.lustre-MDT0001-mdc-*.mds_server_uuid in FULL state after 0 sec [ 1277.553561] Lustre: server umount lustre-MDT0000 complete [ 1281.270984] LustreError: 43073:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1788639978 with bad export cookie 16857562073661317695 [ 1281.285315] LustreError: MGC192.168.203.150@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 1281.290224] LustreError: 43073:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 1281.638084] Lustre: server umount lustre-MDT0001 complete [ 1295.198207] Lustre: server umount lustre-OST0000 complete [ 1298.399173] Lustre: 3647:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1788639979/real 1788639979] req@ffff99a007f12a00 x1875523703060352/t0(0) o400->lustre-MDT0001-lwp-OST0001@0@lo:12/10 lens 224/224 e 0 to 1 dl 1788639995 ref 1 fl Rpc:XNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 [ 1299.273538] Lustre: server umount lustre-OST0001 complete [ 1309.049412] Lustre: DEBUG MARKER: == conf-sanity test 76d: verify llite.*.xattr_cache can be set by 'lctl set_param -P' correctly ========================================================== 16:26:44 (1788640004) [ 1327.837451] Lustre: DEBUG MARKER: oleg350-server.virtnet: executing load_modules_local [ 1338.883081] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1344.044468] Lustre: DEBUG MARKER: oleg350-server.virtnet: executing set_default_debug -1 all [ 1353.200347] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1358.698226] Lustre: DEBUG MARKER: oleg350-server.virtnet: executing set_default_debug -1 all [ 1361.738334] Lustre: 48830:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify general/debug_raw_pointers=Y (mode = 0) failed: rc = -17 [ 1369.046631] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 1376.332943] Lustre: DEBUG MARKER: oleg350-server.virtnet: executing set_default_debug -1 all [ 1384.535500] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 1391.185224] Lustre: DEBUG MARKER: oleg350-server.virtnet: executing set_default_debug -1 all [ 1398.217316] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 1402.146127] Lustre: Setting parameter general.llite.*.xattr_cache=0 in log params [ 1414.736455] Lustre: server umount lustre-MDT0000 complete [ 1415.142323] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 1418.349757] LustreError: 47663:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1788640115 with bad export cookie 16857562073661318521 [ 1418.356429] LustreError: MGC192.168.203.150@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 1418.362919] LustreError: 47663:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 1418.717882] Lustre: server umount lustre-MDT0001 complete [ 1432.626964] Lustre: server umount lustre-OST0000 complete [ 1446.809598] Lustre: server umount lustre-OST0001 complete [ 1456.820151] Lustre: DEBUG MARKER: == conf-sanity test 77: comma-separated MGS NIDs and failover node NIDs ========================================================== 16:29:12 (1788640152) [ 1459.380122] Lustre: DEBUG MARKER: SKIP: conf-sanity test_77 mixed loopback and real device not working [ 1461.360418] Lustre: DEBUG MARKER: == conf-sanity test 78: run resize2fs on MDT and OST filesystems ========================================================== 16:29:16 (1788640156) [ 1474.824312] Lustre: DEBUG MARKER: oleg350-server.virtnet: executing set_hostid [ 1484.244330] Lustre: DEBUG MARKER: oleg350-server.virtnet: executing load_modules_local [ 1507.086123] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1507.272549] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 1507.349295] Lustre: lustre-MDT0000: new disk, initializing [ 1507.412040] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 1510.959612] Lustre: DEBUG MARKER: oleg350-server.virtnet: executing set_default_debug -1 all [ 1519.731294] Lustre: DEBUG MARKER: oleg350-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 1525.615416] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 1525.807654] Lustre: lustre-OST0000: new disk, initializing [ 1525.812188] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 1525.819512] Lustre: Skipped 1 previous similar message [ 1525.825982] Lustre: 54904:0:(osd_compat.c:1353:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 1527.912064] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:0:ost [ 1527.923544] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:0:ost] [ 1527.960761] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x240000400 [ 1531.649752] Lustre: DEBUG MARKER: oleg350-server.virtnet: executing set_default_debug -1 all [ 1539.909238] Lustre: DEBUG MARKER: oleg350-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 1541.895884] Lustre: DEBUG MARKER: create test files [ 1578.976227] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 1578.990517] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 1579.018300] Lustre: Skipped 14 previous similar messages [ 1579.036882] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 1579.042151] Lustre: Skipped 17 previous similar messages [ 1584.982806] Lustre: server umount lustre-OST0000 complete [ 1589.176742] Lustre: server umount lustre-MDT0000 complete [ 1600.451886] Lustre: DEBUG MARKER: oleg350-server.virtnet: executing unload_modules_local [ 1603.639708] Key type lgssc unregistered [ 1603.949385] LNet: 56533:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 1603.953439] LNetError: 56533:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 1603.983075] LNet: Removed LNI 192.168.203.150@tcp [ 1605.193511] Key type .llcrypt unregistered [ 1605.195311] Key type ._llcrypt unregistered [ 1632.727308] Key type ._llcrypt registered [ 1632.734830] Key type .llcrypt registered [ 1632.878134] Lustre: DEBUG MARKER: oleg350-server.virtnet: executing load_modules_local [ 1633.953035] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 1633.973084] alg: No test for adler32 (adler32-zlib) [ 1635.038843] Lustre: Lustre: Build Version: 2.17.58_3_g980e1e2 [ 1635.245734] LNet: Added LNI 192.168.203.150@tcp [8/256/0/180] [ 1636.887214] Key type lgssc registered [ 1637.905590] Lustre: Echo OBD driver; http://www.lustre.org/ [ 1645.594873] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 1645.609464] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1646.992137] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 1652.515318] Lustre: DEBUG MARKER: oleg350-server.virtnet: executing set_default_debug -1 all [ 1661.814996] Lustre: DEBUG MARKER: oleg350-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 1670.273263] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 1670.711633] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 1671.772472] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x240000400:99 to 0x240000400:129) [ 1671.976282] Lustre: 59246:0:(osd_internal.h:1460:osd_trans_exec_op()) lustre-OST0000: opcode 8: before 515 < left 544, rollback = 9 [ 1671.983851] Lustre: 59246:0:(osd_handler.c:2090:osd_trans_dump_creds()) create: 32/128/0, destroy: 0/0/0 [ 1671.990968] Lustre: 59246:0:(osd_handler.c:2097:osd_trans_dump_creds()) attr_set: 0/0/0, xattr_set: 0/0/0 [ 1671.997761] Lustre: 59246:0:(osd_handler.c:2107:osd_trans_dump_creds()) write: 2/18/1, punch: 0/0/0, quota 1/3/0 [ 1672.004196] Lustre: 59246:0:(osd_handler.c:2114:osd_trans_dump_creds()) insert: 32/544/0, delete: 0/0/0 [ 1672.009958] Lustre: 59246:0:(osd_handler.c:2121:osd_trans_dump_creds()) ref_add: 0/0/0, ref_del: 0/0/0 [ 1678.537284] Lustre: DEBUG MARKER: oleg350-server.virtnet: executing set_default_debug -1 all [ 1688.050146] Lustre: DEBUG MARKER: oleg350-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 1695.512948] Lustre: DEBUG MARKER: check files after expanding the MDT and OST filesystems [ 1702.590606] Lustre: DEBUG MARKER: create more files after expanding the MDT and OST filesystems [ 1712.619075] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 1712.624309] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 1712.628298] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 1717.729902] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 1718.480113] Lustre: server umount lustre-OST0000 complete [ 1722.752734] Lustre: server umount lustre-MDT0000 complete [ 1735.733865] Lustre: DEBUG MARKER: oleg350-server.virtnet: executing unload_modules_local [ 1738.758372] Key type lgssc unregistered [ 1739.085275] LNet: 60856:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 1739.093788] LNetError: 60856:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 1740.134703] LNet: Removed LNI 192.168.203.150@tcp [ 1741.122570] Key type .llcrypt unregistered [ 1741.126368] Key type ._llcrypt unregistered [ 1776.378481] Key type ._llcrypt registered [ 1776.381663] Key type .llcrypt registered [ 1776.554641] Lustre: DEBUG MARKER: oleg350-server.virtnet: executing load_modules_local [ 1778.290110] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 1778.312890] alg: No test for adler32 (adler32-zlib) [ 1779.395686] Lustre: Lustre: Build Version: 2.17.58_3_g980e1e2 [ 1779.657599] LNet: Added LNI 192.168.203.150@tcp [8/256/0/180] [ 1781.368393] Key type lgssc registered [ 1782.710844] Lustre: Echo OBD driver; http://www.lustre.org/ [ 1790.745549] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 1790.795532] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1792.275224] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 1797.014242] Lustre: DEBUG MARKER: oleg350-server.virtnet: executing set_default_debug -1 all [ 1805.026321] Lustre: DEBUG MARKER: oleg350-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 1811.834813] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 1812.267461] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 1817.398272] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x240000400:140 to 0x240000400:161) [ 1817.521159] Lustre: 63699:0:(osd_internal.h:1460:osd_trans_exec_op()) lustre-OST0000: opcode 8: before 509 < left 544, rollback = 9 [ 1817.543808] Lustre: 63699:0:(osd_handler.c:2090:osd_trans_dump_creds()) create: 32/128/6, destroy: 0/0/0 [ 1817.560663] Lustre: 63699:0:(osd_handler.c:2097:osd_trans_dump_creds()) attr_set: 0/0/0, xattr_set: 0/0/0 [ 1817.574425] Lustre: 63699:0:(osd_handler.c:2107:osd_trans_dump_creds()) write: 2/18/1, punch: 0/0/0, quota 1/3/0 [ 1817.592343] Lustre: 63699:0:(osd_handler.c:2114:osd_trans_dump_creds()) insert: 32/544/0, delete: 0/0/0 [ 1817.606270] Lustre: 63699:0:(osd_handler.c:2121:osd_trans_dump_creds()) ref_add: 0/0/0, ref_del: 0/0/0 [ 1818.306753] Lustre: DEBUG MARKER: oleg350-server.virtnet: executing set_default_debug -1 all [ 1825.671627] Lustre: DEBUG MARKER: oleg350-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 1830.832663] Lustre: DEBUG MARKER: check files after shrinking the MDT and OST filesystems [ 1843.168831] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 1843.178456] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 1843.208154] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 1845.885617] Lustre: server umount lustre-OST0000 complete [ 1850.080286] Lustre: server umount lustre-MDT0000 complete [ 1865.511389] Lustre: DEBUG MARKER: oleg350-server.virtnet: executing unload_modules_local [ 1868.138730] Key type lgssc unregistered [ 1868.412292] LNet: 65259:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 1868.416948] LNetError: 65259:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 1869.477795] LNet: Removed LNI 192.168.203.150@tcp [ 1870.399383] Key type .llcrypt unregistered [ 1870.403511] Key type ._llcrypt unregistered [ 1886.976697] Key type ._llcrypt registered [ 1886.979397] Key type .llcrypt registered [ 1887.099727] Lustre: DEBUG MARKER: oleg350-server.virtnet: executing set_hostid [ 1902.345483] Lustre: DEBUG MARKER: oleg350-server.virtnet: executing load_modules_local [ 1903.702177] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 1903.741666] alg: No test for adler32 (adler32-zlib) [ 1904.870411] Lustre: Lustre: Build Version: 2.17.58_3_g980e1e2 [ 1905.104199] LNet: Added LNI 192.168.203.150@tcp [8/256/0/180] [ 1906.823252] Key type lgssc registered [ 1908.045371] Lustre: Echo OBD driver; http://www.lustre.org/ [ 1945.786253] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 1945.823762] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1947.131868] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 1947.175598] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 1947.276643] Lustre: lustre-MDT0000: new disk, initializing [ 1947.374064] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 1947.401360] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 1952.227644] Lustre: DEBUG MARKER: oleg350-server.virtnet: executing set_default_debug -1 all [ 1963.173405] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1963.315803] Lustre: 68406:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 1963.389377] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [ 1963.393419] Lustre: Skipped 1 previous similar message [ 1963.492825] Lustre: lustre-MDT0001: new disk, initializing [ 1963.611132] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 1963.655640] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [ 1963.662834] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 1968.074600] Lustre: DEBUG MARKER: oleg350-server.virtnet: executing set_default_debug -1 all [ 1977.351900] Lustre: DEBUG MARKER: oleg350-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 1983.163093] Lustre: DEBUG MARKER: oleg350-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 1991.349921] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 1991.703119] Lustre: lustre-OST0000: new disk, initializing [ 1991.707174] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 1991.722214] Lustre: 70375:0:(osd_compat.c:1353:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 1991.825792] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 1999.231922] Lustre: DEBUG MARKER: oleg350-server.virtnet: executing set_default_debug -1 all [ 2000.551286] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [ 2000.577170] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 2000.752432] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 2010.995698] Lustre: DEBUG MARKER: oleg350-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 2016.758842] Lustre: DEBUG MARKER: oleg350-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 [ 2016.929128] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [ 2022.034291] Lustre: DEBUG MARKER: oleg350-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid 50 [ 2022.324899] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [ 2026.464938] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 2026.483058] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2026.486797] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 2026.499799] Lustre: Skipped 1 previous similar message [ 2030.635639] Lustre: server umount lustre-OST0000 complete [ 2035.169202] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 2035.187762] LustreError: Skipped 1 previous similar message [ 2035.197073] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2035.211025] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 2035.220427] Lustre: Skipped 1 previous similar message [ 2036.708635] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 2036.710435] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2041.237586] Lustre: server umount lustre-MDT0000 complete [ 2045.313230] LustreError: 68399:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1788640742 with bad export cookie 8617681313403635646 [ 2045.316030] LustreError: MGC192.168.203.150@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 2045.333513] LustreError: 68399:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 1 previous similar message [ 2045.823337] Lustre: server umount lustre-MDT0001 complete [ 2055.177799] Lustre: DEBUG MARKER: == conf-sanity test 79: format MDT/OST without mgs option (should return errors) ========================================================== 16:39:10 (1788640750) [ 2065.946146] Lustre: DEBUG MARKER: oleg350-server.virtnet: executing load_modules_local [ 2107.544442] Lustre: DEBUG MARKER: oleg350-server.virtnet: executing set_hostid [ 2117.927279] Lustre: DEBUG MARKER: oleg350-server.virtnet: executing load_modules_local [ 2159.616282] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 2159.839778] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 2159.879930] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 2159.996824] Lustre: lustre-MDT0000: new disk, initializing [ 2160.150809] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 2160.183988] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 2165.205930] Lustre: DEBUG MARKER: oleg350-server.virtnet: executing set_default_debug -1 all [ 2178.057115] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 2178.132580] Lustre: 76398:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 2178.186503] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [ 2178.189906] Lustre: Skipped 1 previous similar message [ 2178.273832] Lustre: lustre-MDT0001: new disk, initializing [ 2178.475306] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 2178.598123] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [ 2178.618228] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 2183.529187] Lustre: DEBUG MARKER: oleg350-server.virtnet: executing set_default_debug -1 all [ 2195.800920] Lustre: DEBUG MARKER: oleg350-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 2202.698431] Lustre: DEBUG MARKER: oleg350-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 2210.966751] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 2211.212787] Lustre: lustre-OST0000: new disk, initializing [ 2211.217067] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 2211.228901] Lustre: 78371:0:(osd_compat.c:1353:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 2211.323418] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 2218.275906] Lustre: DEBUG MARKER: oleg350-server.virtnet: executing set_default_debug -1 all [ 2221.131973] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [ 2221.144309] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 2221.247542] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 2227.829429] Lustre: DEBUG MARKER: oleg350-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 2232.785434] Lustre: DEBUG MARKER: oleg350-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 [ 2233.055392] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [ 2237.049236] Lustre: DEBUG MARKER: oleg350-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid 50 [ 2237.234809] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [ 2242.022360] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 2242.022923] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2242.024729] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 2242.024736] Lustre: Skipped 1 previous similar message [ 2242.030321] LustreError: Skipped 1 previous similar message [ 2242.057485] Lustre: Skipped 1 previous similar message [ 2245.481390] Lustre: server umount lustre-OST0000 complete [ 2250.212590] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 2250.221218] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2250.240291] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 2250.248131] Lustre: Skipped 1 previous similar message [ 2255.352831] Lustre: server umount lustre-MDT0000 complete [ 2257.377288] LustreError: 76411:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 2257.395956] LustreError: 76411:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 4 previous similar messages [ 2258.561976] LustreError: 76392:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1788640955 with bad export cookie 8617681313403636528 [ 2258.564328] LustreError: MGC192.168.203.150@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 2258.570419] LustreError: 76392:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 3 previous similar messages [ 2258.779228] Lustre: server umount lustre-MDT0001 complete [ 2266.826421] Lustre: DEBUG MARKER: == conf-sanity test 80: mgc import reconnect race ======== 16:42:42 (1788640962) [ 2274.960126] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 2275.374446] LustreError: 80781:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 2275.505345] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 2279.961571] Lustre: DEBUG MARKER: oleg350-server.virtnet: executing set_default_debug -1 all [ 2280.930213] LustreError: 80782:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 2286.050162] LustreError: 80781:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 2289.123367] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 2293.596776] Lustre: DEBUG MARKER: oleg350-server.virtnet: executing set_default_debug -1 all [ 2300.430775] Lustre: DEBUG MARKER: oleg350-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 2305.397950] Lustre: DEBUG MARKER: oleg350-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 2313.125881] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 2313.603838] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 2313.608891] Lustre: Skipped 1 previous similar message [ 2319.447985] Lustre: DEBUG MARKER: oleg350-server.virtnet: executing set_default_debug -1 all [ 2326.958984] Lustre: DEBUG MARKER: oleg350-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 2330.989492] Lustre: 83452:0:(genops.c:1774:obd_export_evict_by_uuid()) MGS: evicting 2045ac1e-b52b-4961-ab57-93214692fe91 at adminstrative request [ 2331.105888] LustreError: MGC192.168.203.150@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 2331.142346] Lustre: *** cfs_fail_loc=906, val=2147483648*** [ 2341.360176] Lustre: MGS: Client 2045ac1e-b52b-4961-ab57-93214692fe91 (at 0@lo) reconnecting [ 2341.375567] Lustre: Evicted from MGS (at 0@lo) after server handle changed from 0x7798277f00f20ea2 to 0x7798277f00f21142 [ 2341.394190] Lustre: MGC192.168.203.150@tcp: Connection restored to 0@lo (at 0@lo) [ 2369.399453] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 2369.589851] Lustre: lustre-OST0001: new disk, initializing [ 2369.593880] Lustre: srv-lustre-OST0001: No data found on store. Initialize space. [ 2369.600696] Lustre: 83843:0:(osd_compat.c:1353:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 2376.480542] Lustre: DEBUG MARKER: oleg350-server.virtnet: executing set_default_debug -1 all [ 2379.880644] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x00000002c0000400-0x0000000300000400]:1:ost [ 2379.889619] Lustre: cli-lustre-OST0001-super: Allocated super-sequence [0x00000002c0000400-0x0000000300000400]:1:ost] [ 2380.003456] Lustre: lustre-OST0001-osc-MDT0000: update sequence from 0x100010000 to 0x2c0000401 [ 2386.183611] Lustre: DEBUG MARKER: oleg350-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0001-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 2390.497051] LustreError: lustre-OST0001-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 2390.500683] Lustre: lustre-OST0001-osc-MDT0000: Connection to lustre-OST0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2390.520648] Lustre: lustre-OST0001: Not available for connect from 0@lo (stopping) [ 2390.541245] Lustre: Skipped 2 previous similar messages [ 2390.548539] Lustre: Skipped 2 previous similar messages [ 2403.296776] Lustre: lustre-OST0001 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 2403.400169] Lustre: server umount lustre-OST0001 complete [ 2405.866523] LustreError: 82602:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 2405.886578] LustreError: 82602:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 1 previous similar message [ 2407.904534] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 2407.911426] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2407.929044] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 2407.935333] Lustre: Skipped 5 previous similar messages [ 2416.135014] LustreError: 82601:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 2416.163210] LustreError: 82601:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 3 previous similar messages [ 2422.239218] Lustre: lustre-OST0000 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 2422.329588] Lustre: server umount lustre-OST0000 complete [ 2431.992854] Lustre: server umount lustre-MDT0000 complete [ 2435.924215] LustreError: 80761:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1788641132 with bad export cookie 8617681313403638082 [ 2435.934117] LustreError: MGC192.168.203.150@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 2436.272126] Lustre: server umount lustre-MDT0001 complete [ 2444.155454] Lustre: DEBUG MARKER: == conf-sanity test 81: sparse OST indexing ============== 16:45:39 (1788641139) [ 2445.768395] Lustre: DEBUG MARKER: SKIP: conf-sanity test_81 needs >= 3 OSTs [ 2447.261048] Lustre: DEBUG MARKER: == conf-sanity test 82a: specify OSTs for file (succeed) or directory (succeed) ========================================================== 16:45:43 (1788641143) [ 2448.644365] Lustre: DEBUG MARKER: SKIP: conf-sanity test_82a needs >= 3 OSTs [ 2450.478738] Lustre: DEBUG MARKER: == conf-sanity test 82b: specify OSTs for file with --pool and --ost-list options ========================================================== 16:45:46 (1788641146) [ 2452.239818] Lustre: DEBUG MARKER: SKIP: conf-sanity test_82b needs >= 4 OSTs [ 2454.168058] Lustre: DEBUG MARKER: == conf-sanity test 82c: specify sparse OSTs for setstripe (should not crash) ========================================================== 16:45:49 (1788641149) [ 2456.010112] Lustre: DEBUG MARKER: SKIP: conf-sanity test_82c needs >= 4 OSTs [ 2458.280583] Lustre: DEBUG MARKER: == conf-sanity test 83: ENOSPACE on OST doesn't cause message VFS: Busy inodes after unmount ... ========================================================== 16:45:53 (1788641153) [ 2460.254635] Lustre: DEBUG MARKER: mount the OST /dev/mapper/ost1_flakey as a ldiskfs filesystem [ 2467.558927] loop: module loaded [ 2467.595666] loop0: detected capacity change from 0 to 4294967296 [ 2467.668573] LDISKFS-fs (loop0): mounted filesystem with ordered data mode. Opts: (null) [ 2470.188942] Lustre: DEBUG MARKER: run llverfs in partial mode on the OST ldiskfs /mnt/lustre-ost1 [ 2473.882692] Lustre: DEBUG MARKER: oleg350-server.virtnet: executing run_llverfs /mnt/lustre-ost1 -vpl no [ 2478.735442] Lustre: DEBUG MARKER: unmount the OST /dev/mapper/ost1_flakey [ 2505.622618] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 2505.694090] LustreError: 87970:0:(obd_config.c:845:class_setup()) setup lustre-OST0000-osd failed (-28) [ 2505.704213] LustreError: 87970:0:(obd_mount.c:259:lustre_start_simple()) lustre-OST0000-osd setup error -28 [ 2505.709557] LustreError: 87970:0:(tgt_mount.c:2585:server_fill_super()) Unable to start osd on /dev/mapper/ost1_flakey: -28 [ 2505.714943] LustreError: 87970:0:(super25.c:179:lustre_fill_super()) llite: Unable to mount : rc = -28 [ 2525.805267] Lustre: DEBUG MARKER: oleg350-server.virtnet: executing set_hostid [ 2536.884575] Lustre: DEBUG MARKER: oleg350-server.virtnet: executing load_modules_local [ 2576.239270] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 2576.509779] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 2576.557937] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 2576.677247] Lustre: lustre-MDT0000: new disk, initializing [ 2576.824784] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 2576.834775] Lustre: Skipped 1 previous similar message [ 2576.868780] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 2582.186431] Lustre: DEBUG MARKER: oleg350-server.virtnet: executing set_default_debug -1 all [ 2593.472783] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 2593.623669] Lustre: 90837:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 2593.946784] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 2598.228468] Lustre: DEBUG MARKER: oleg350-server.virtnet: executing set_default_debug -1 all [ 2607.842194] Lustre: DEBUG MARKER: oleg350-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 2613.367452] Lustre: DEBUG MARKER: oleg350-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 2620.717522] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 2621.028494] Lustre: 92809:0:(osd_compat.c:1353:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 2627.367817] Lustre: DEBUG MARKER: oleg350-server.virtnet: executing set_default_debug -1 all [ 2628.157744] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 2628.355856] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 2637.722829] Lustre: DEBUG MARKER: oleg350-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 2642.779277] Lustre: DEBUG MARKER: oleg350-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 [ 2643.053717] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [ 2647.473847] Lustre: DEBUG MARKER: oleg350-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid 50 [ 2647.647595] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [ 2654.189337] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2654.194123] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 2654.204176] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 2654.204187] Lustre: Skipped 10 previous similar messages [ 2654.215889] Lustre: Skipped 4 previous similar messages [ 2654.241178] LustreError: Skipped 1 previous similar message [ 2656.011876] Lustre: server umount lustre-OST0000 complete [ 2666.255299] Lustre: server umount lustre-MDT0000 complete [ 2669.536455] LustreError: 90845:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 2669.552174] LustreError: 90845:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 3 previous similar messages [ 2670.037454] LustreError: 90832:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1788641366 with bad export cookie 8617681313403638698 [ 2670.042352] LustreError: MGC192.168.203.150@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 2670.049897] LustreError: 90832:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 5 previous similar messages [ 2670.437906] Lustre: server umount lustre-MDT0001 complete [ 2679.461872] Lustre: DEBUG MARKER: == conf-sanity test 84: check recovery_hard_time ========= 16:49:34 (1788641374) [ 2686.955920] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 2692.179407] Lustre: DEBUG MARKER: oleg350-server.virtnet: executing set_default_debug -1 all [ 2700.483648] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 2705.363867] Lustre: DEBUG MARKER: oleg350-server.virtnet: executing set_default_debug -1 all [ 2714.043590] Lustre: DEBUG MARKER: oleg350-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 2719.804310] Lustre: DEBUG MARKER: oleg350-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 2726.384361] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 2726.612484] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 2726.620227] Lustre: Skipped 4 previous similar messages [ 2732.521215] Lustre: DEBUG MARKER: oleg350-server.virtnet: executing set_default_debug -1 all [ 2740.505367] Lustre: DEBUG MARKER: oleg350-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 2745.752269] Lustre: DEBUG MARKER: oleg350-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 [ 2745.930352] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [ 2750.201829] Lustre: DEBUG MARKER: oleg350-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid 50 [ 2750.433309] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [ 2756.630779] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 2756.834276] Lustre: lustre-OST0001: new disk, initializing [ 2756.843503] Lustre: Skipped 2 previous similar messages [ 2756.850321] Lustre: srv-lustre-OST0001: No data found on store. Initialize space. [ 2756.861264] Lustre: Skipped 3 previous similar messages [ 2756.871682] Lustre: 98592:0:(osd_compat.c:1353:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 2763.287792] Lustre: DEBUG MARKER: oleg350-server.virtnet: executing set_default_debug -1 all [ 2764.882635] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x00000002c0000400-0x0000000300000400]:1:ost [ 2764.890910] Lustre: Skipped 2 previous similar messages [ 2764.896932] Lustre: cli-lustre-OST0001-super: Allocated super-sequence [0x00000002c0000400-0x0000000300000400]:1:ost] [ 2764.991285] Lustre: lustre-OST0001-osc-MDT0000: update sequence from 0x100010000 to 0x2c0000401 [ 2772.605502] Lustre: DEBUG MARKER: oleg350-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0001-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 2777.389720] Lustre: DEBUG MARKER: oleg350-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0001-osc-MDT0000.ost_server_uuid 50 [ 2777.799098] Lustre: DEBUG MARKER: os[cp].lustre-OST0001-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [ 2782.804896] Lustre: DEBUG MARKER: oleg350-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0001-osc-MDT0001.ost_server_uuid 50 [ 2783.132825] Lustre: DEBUG MARKER: os[cp].lustre-OST0001-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [ 2804.168849] Lustre: DEBUG MARKER: mds1 REPLAY BARRIER on lustre-MDT0000 [ 2830.505158] Lustre: Failing over lustre-MDT0000 [ 2830.916149] Lustre: server umount lustre-MDT0000 complete [ 2831.501412] LustreError: 95216:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0000: not available for connect from 192.168.203.50@tcp (no target). If you are running an HA pair check that the target is mounted on the other server. [ 2831.514277] LustreError: 95216:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 4 previous similar messages [ 2831.844156] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2831.869957] Lustre: Skipped 4 previous similar messages [ 2847.200852] Lustre: 66631:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1788641528/real 1788641528] req@00000000bff70991 x1875525608158464/t0(0) o400->MGC192.168.203.150@tcp@0@lo:26/25 lens 224/224 e 0 to 1 dl 1788641544 ref 1 fl Rpc:XNQr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 [ 2847.232406] LustreError: MGC192.168.203.150@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 2854.033404] LDISKFS-fs (dm-0): 4 truncates cleaned up [ 2854.035966] LDISKFS-fs (dm-0): recovery complete [ 2854.054559] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 2857.449627] Lustre: Evicted from MGS (at 0@lo) after server handle changed from 0x0 to 0x7798277f00f2dbbd [ 2857.462397] Lustre: MGC192.168.203.150@tcp: Connection restored to 0@lo (at 0@lo) [ 2857.797808] Lustre: lustre-MDT0000: in recovery but waiting for the first client to connect [ 2860.083829] Lustre: lustre-MDT0000: Will be in recovery for at least 1:00, or until 3 clients reconnect [ 2860.084445] Lustre: 96462:0:(ldlm_lib.c:2110:extend_recovery_timer()) lustre-MDT0000: extended recovery timer reached hard limit: 60, extend: 0 [ 2862.139429] Lustre: DEBUG MARKER: oleg350-server.virtnet: executing set_default_debug -1 all [ 2863.074185] Lustre: 95948:0:(ldlm_lib.c:2110:extend_recovery_timer()) lustre-MDT0000: extended recovery timer reached hard limit: 60, extend: 0 [ 2863.087148] Lustre: 95948:0:(ldlm_lib.c:2110:extend_recovery_timer()) Skipped 1 previous similar message [ 2863.125443] Lustre: lustre-MDT0000-lwp-MDT0001: Connection restored to 0@lo (at 0@lo) [ 2863.192301] LustreError: 101440:0:(ldlm_lib.c:2723:replay_request_or_update()) cfs_fail_timeout id 709 sleeping for 300ms [ 2863.525411] LustreError: 101440:0:(ldlm_lib.c:2723:replay_request_or_update()) cfs_fail_timeout id 709 awake [ 2863.858616] LustreError: 101440:0:(ldlm_lib.c:2723:replay_request_or_update()) cfs_fail_timeout id 709 sleeping for 300ms [ 2863.875555] LustreError: 101440:0:(ldlm_lib.c:2723:replay_request_or_update()) Skipped 1 previous similar message [ 2864.201209] LustreError: 101440:0:(ldlm_lib.c:2723:replay_request_or_update()) cfs_fail_timeout id 709 awake [ 2864.210567] LustreError: 101440:0:(ldlm_lib.c:2723:replay_request_or_update()) Skipped 1 previous similar message [ 2864.223916] Lustre: 101440:0:(ldlm_lib.c:2110:extend_recovery_timer()) lustre-MDT0000: extended recovery timer reached hard limit: 60, extend: 1 [ 2864.234082] Lustre: 101440:0:(ldlm_lib.c:2110:extend_recovery_timer()) Skipped 7 previous similar messages [ 2864.882394] LustreError: 101440:0:(ldlm_lib.c:2723:replay_request_or_update()) cfs_fail_timeout id 709 sleeping for 300ms [ 2864.901392] LustreError: 101440:0:(ldlm_lib.c:2723:replay_request_or_update()) Skipped 2 previous similar messages [ 2865.231494] LustreError: 101440:0:(ldlm_lib.c:2723:replay_request_or_update()) cfs_fail_timeout id 709 awake [ 2865.238574] LustreError: 101440:0:(ldlm_lib.c:2723:replay_request_or_update()) Skipped 2 previous similar messages [ 2866.531265] Lustre: 101440:0:(ldlm_lib.c:2110:extend_recovery_timer()) lustre-MDT0000: extended recovery timer reached hard limit: 60, extend: 1 [ 2866.547495] Lustre: 101440:0:(ldlm_lib.c:2110:extend_recovery_timer()) Skipped 6 previous similar messages [ 2867.195449] LustreError: 101440:0:(ldlm_lib.c:2723:replay_request_or_update()) cfs_fail_timeout id 709 sleeping for 300ms [ 2867.207585] LustreError: 101440:0:(ldlm_lib.c:2723:replay_request_or_update()) Skipped 6 previous similar messages [ 2867.535130] LustreError: 101440:0:(ldlm_lib.c:2723:replay_request_or_update()) cfs_fail_timeout id 709 awake [ 2867.545885] LustreError: 101440:0:(ldlm_lib.c:2723:replay_request_or_update()) Skipped 6 previous similar messages [ 2870.711921] Lustre: 101440:0:(ldlm_lib.c:2110:extend_recovery_timer()) lustre-MDT0000: extended recovery timer reached hard limit: 60, extend: 1 [ 2870.728339] Lustre: 101440:0:(ldlm_lib.c:2110:extend_recovery_timer()) Skipped 12 previous similar messages [ 2871.376564] LustreError: 101440:0:(ldlm_lib.c:2723:replay_request_or_update()) cfs_fail_timeout id 709 sleeping for 300ms [ 2871.396467] LustreError: 101440:0:(ldlm_lib.c:2723:replay_request_or_update()) Skipped 12 previous similar messages [ 2871.711138] LustreError: 101440:0:(ldlm_lib.c:2723:replay_request_or_update()) cfs_fail_timeout id 709 awake [ 2871.715735] LustreError: 101440:0:(ldlm_lib.c:2723:replay_request_or_update()) Skipped 12 previous similar messages [ 2878.992291] Lustre: 101440:0:(ldlm_lib.c:2110:extend_recovery_timer()) lustre-MDT0000: extended recovery timer reached hard limit: 60, extend: 1 [ 2879.006090] Lustre: 101440:0:(ldlm_lib.c:2110:extend_recovery_timer()) Skipped 25 previous similar messages [ 2879.673729] LustreError: 101440:0:(ldlm_lib.c:2723:replay_request_or_update()) cfs_fail_timeout id 709 sleeping for 300ms [ 2879.691508] LustreError: 101440:0:(ldlm_lib.c:2723:replay_request_or_update()) Skipped 25 previous similar messages [ 2880.015118] LustreError: 101440:0:(ldlm_lib.c:2723:replay_request_or_update()) cfs_fail_timeout id 709 awake [ 2880.028445] LustreError: 101440:0:(ldlm_lib.c:2723:replay_request_or_update()) Skipped 25 previous similar messages [ 2895.248818] Lustre: 101440:0:(ldlm_lib.c:2110:extend_recovery_timer()) lustre-MDT0000: extended recovery timer reached hard limit: 60, extend: 1 [ 2895.267086] Lustre: 101440:0:(ldlm_lib.c:2110:extend_recovery_timer()) Skipped 50 previous similar messages [ 2895.907099] LustreError: 101440:0:(ldlm_lib.c:2723:replay_request_or_update()) cfs_fail_timeout id 709 sleeping for 300ms [ 2895.918771] LustreError: 101440:0:(ldlm_lib.c:2723:replay_request_or_update()) Skipped 50 previous similar messages [ 2896.243565] LustreError: 101440:0:(ldlm_lib.c:2723:replay_request_or_update()) cfs_fail_timeout id 709 awake [ 2896.250123] LustreError: 101440:0:(ldlm_lib.c:2723:replay_request_or_update()) Skipped 50 previous similar messages [ 2920.065558] Lustre: 101440:0:(ldlm_lib.c:2419:target_recovery_overseer()) lustre-MDT0000 recovery is aborted by hard timeout [ 2920.080044] Lustre: 101440:0:(ldlm_lib.c:2429:target_recovery_overseer()) recovery is aborted, evict exports in recovery [ 2920.099877] Lustre: 101440:0:(genops.c:1601:class_disconnect_stale_exports()) lustre-MDT0000: disconnect stale client 81b79715-1d40-445f-8796-4fa23555b587@192.168.203.50@tcp [ 2920.117104] Lustre: lustre-MDT0000: disconnecting 1 stale clients [ 2920.132492] Lustre: 101440:0:(ldlm_lib.c:1939:abort_req_replay_queue()) @@@ aborted: req@00000000044e444d x1875525592225024/t0(8589934775) o101->81b79715-1d40-445f-8796-4fa23555b587@192.168.203.50@tcp:572/0 lens 592/0 e 0 to 0 dl 1788641627 ref 1 fl Complete:/604/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:0 [ 2920.163613] Lustre: lustre-MDT0000-osd: cancel update llog [0x200000400:0x1:0x0] [ 2920.176207] Lustre: lustre-MDT0000: Denying connection for new client 81b79715-1d40-445f-8796-4fa23555b587 (at 192.168.203.50@tcp), waiting for 3 known clients (2 recovered, 0 in progress, and 1 evicted) to recover in 0:00 [ 2920.216131] Lustre: lustre-MDT0001-osp-MDT0000: cancel update llog [0x240000401:0x1:0x0] [ 2920.256735] Lustre: lustre-MDT0000-osp-MDT0001: Connection restored to 0@lo (at 0@lo) [ 2920.259796] Lustre: lustre-MDT0000: Recovery over after 1:00, of 3 clients 2 recovered and 1 was evicted. [ 2920.263663] Lustre: Skipped 2 previous similar messages [ 2920.309204] Lustre: lustre-OST0001: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x2c0000401:47 to 0x2c0000401:545) [ 2920.309470] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:78 to 0x280000401:577) [ 2934.769486] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 2934.790081] Lustre: Skipped 5 previous similar messages [ 2949.087654] Lustre: lustre-OST0000 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 2949.292788] Lustre: server umount lustre-OST0000 complete [ 2950.124938] LustreError: 97043:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-OST0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 2950.138492] LustreError: 97043:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 37 previous similar messages [ 2954.721140] LustreError: lustre-OST0001-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 2954.727527] LustreError: Skipped 1 previous similar message [ 2959.335861] Lustre: server umount lustre-OST0001 complete [ 2965.476871] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2965.490125] Lustre: Skipped 5 previous similar messages [ 2969.700189] Lustre: server umount lustre-MDT0000 complete [ 2973.201627] LustreError: 95200:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1788641670 with bad export cookie 8617681313403689917 [ 2973.202258] LustreError: MGC192.168.203.150@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 2973.209075] LustreError: 95200:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 2 previous similar messages [ 2973.517735] Lustre: server umount lustre-MDT0001 complete [ 2982.118643] Lustre: DEBUG MARKER: == conf-sanity test 85: osd_ost init: fail ea_fid_set ==== 16:54:37 (1788641677) [ 2989.058533] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 2989.063831] Lustre: *** cfs_fail_loc=197, val=0*** [ 3004.639680] LustreError: Failed to get MGS log lustre-sptlrpc and no local copy. [ 3009.759454] LustreError: 103414:0:(mgc_request.c:233:do_config_log_add()) MGC192.168.203.150@tcp: failed processing log, type 4: rc = -110 [ 3014.902204] Lustre: *** cfs_fail_loc=197, val=0*** [ 3014.907258] Lustre: Skipped 65 previous similar messages [ 3035.487258] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 3035.498808] Lustre: Skipped 2 previous similar messages [ 3041.590428] Lustre: DEBUG MARKER: oleg350-server.virtnet: executing set_default_debug -1 all [ 3049.059627] Lustre: DEBUG MARKER: oleg350-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 3051.372955] Lustre: server umount lustre-OST0000 complete [ 3060.169219] Lustre: DEBUG MARKER: == conf-sanity test 86: Replacing mkfs.lustre -G option == 16:55:55 (1788641755) [ 3081.833448] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 3082.302147] LustreError: 105036:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 3082.339899] LustreError: 105036:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 4 previous similar messages [ 3087.223147] Lustre: DEBUG MARKER: oleg350-server.virtnet: executing set_default_debug -1 all [ 3095.072074] Lustre: DEBUG MARKER: oleg350-client.virtnet: executing wait_import_state_mount (FULL|IDLE) mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 3098.658705] LustreError: 105068:0:(osp_object.c:618:osp_attr_get()) lustre-MDT0001-osp-MDT0000: osp_attr_get update error [0x200000009:0x1:0x0]: rc = -5 [ 3098.677749] LustreError: 105068:0:(lod_sub_object.c:917:lod_sub_prep_llog()) lustre-MDT0000-mdtlov: can't get id from catalogs: rc = -5 [ 3098.685880] LustreError: 105068:0:(lod_dev.c:511:lod_sub_recovery_thread()) lustre-MDT0001-osp-MDT0000: get update log duration 16, retries 0, failed: rc = -5 [ 3098.915867] Lustre: server umount lustre-MDT0000 complete [ 3127.476449] Lustre: DEBUG MARKER: oleg350-server.virtnet: executing set_hostid [ 3136.667871] Lustre: DEBUG MARKER: oleg350-server.virtnet: executing load_modules_local [ 3169.591298] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 3169.853938] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 3169.883728] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 3169.994965] Lustre: lustre-MDT0000: new disk, initializing [ 3170.102406] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 3174.315863] Lustre: DEBUG MARKER: oleg350-server.virtnet: executing set_default_debug -1 all [ 3184.095299] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 3184.197604] Lustre: 109243:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 3184.401605] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 3188.289829] Lustre: DEBUG MARKER: oleg350-server.virtnet: executing set_default_debug -1 all [ 3196.763728] Lustre: DEBUG MARKER: oleg350-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 3202.241122] Lustre: DEBUG MARKER: oleg350-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 3209.055284] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 3209.413679] Lustre: 111210:0:(osd_compat.c:1353:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 3214.986370] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 3215.555572] Lustre: DEBUG MARKER: oleg350-server.virtnet: executing set_default_debug -1 all [ 3224.509716] Lustre: DEBUG MARKER: oleg350-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 3228.631816] Lustre: DEBUG MARKER: oleg350-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 [ 3228.900855] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [ 3233.411418] Lustre: DEBUG MARKER: oleg350-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid 50 [ 3233.595314] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [ 3235.300492] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 3235.314904] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 3235.331693] Lustre: Skipped 1 previous similar message [ 3235.338989] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 3235.342130] Lustre: Skipped 9 previous similar messages [ 3241.586918] Lustre: server umount lustre-OST0000 complete [ 3251.857990] Lustre: server umount lustre-MDT0000 complete [ 3255.409032] LustreError: 109234:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1788641952 with bad export cookie 8617681313403700557 [ 3255.429143] LustreError: MGC192.168.203.150@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 3255.432352] LustreError: 109234:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 1 previous similar message [ 3255.736377] Lustre: server umount lustre-MDT0001 complete [ 3263.956324] Lustre: DEBUG MARKER: == conf-sanity test 87: check if MDT inode can hold EAs with N stripes properly ========================================================== 16:59:19 (1788641959) [ 3271.375752] Lustre: DEBUG MARKER: oleg350-server.virtnet: executing unload_modules_local [ 3274.004952] Key type lgssc unregistered [ 3274.400992] LNet: 113694:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 3274.409561] LNetError: 113694:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 3274.426389] LNet: Removed LNI 192.168.203.150@tcp [ 3275.337868] Key type .llcrypt unregistered [ 3275.339749] Key type ._llcrypt unregistered [ 3292.847901] Key type ._llcrypt registered [ 3292.850371] Key type .llcrypt registered [ 3292.946966] Lustre: DEBUG MARKER: oleg350-server.virtnet: executing set_hostid [ 3308.846055] Lustre: DEBUG MARKER: oleg350-server.virtnet: executing load_modules_local [ 3309.923481] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 3310.245922] alg: No test for adler32 (adler32-zlib) [ 3311.343271] Lustre: Lustre: Build Version: 2.17.58_3_g980e1e2 [ 3311.570914] LNet: Added LNI 192.168.203.150@tcp [8/256/0/180] [ 3313.232174] Key type lgssc registered [ 3314.338973] Lustre: Echo OBD driver; http://www.lustre.org/ [ 3357.277688] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 3357.311750] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 3358.692619] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 3358.756660] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 3358.869755] Lustre: lustre-MDT0000: new disk, initializing [ 3358.982289] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 3358.998525] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 3363.450190] Lustre: DEBUG MARKER: oleg350-server.virtnet: executing set_default_debug -1 all [ 3374.486653] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 3374.876979] Lustre: lustre-OST0000: new disk, initializing [ 3374.881118] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 3374.885671] Lustre: Skipped 1 previous similar message [ 3374.889720] Lustre: 117877:0:(osd_compat.c:1353:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 3374.953618] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 3380.356269] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:0:ost [ 3380.375315] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:0:ost] [ 3380.422860] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x240000400 [ 3380.803283] Lustre: DEBUG MARKER: oleg350-server.virtnet: executing set_default_debug -1 all [ 3390.830561] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 3390.995735] Lustre: lustre-OST0001: new disk, initializing [ 3390.998727] Lustre: srv-lustre-OST0001: No data found on store. Initialize space. [ 3391.004186] Lustre: 118739:0:(osd_compat.c:1353:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 3391.064272] Lustre: lustre-OST0001: Imperative Recovery not enabled, recovery window 60-180 [ 3396.851655] Lustre: DEBUG MARKER: oleg350-server.virtnet: executing set_default_debug -1 all [ 3398.302730] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:1:ost [ 3398.311755] Lustre: cli-lustre-OST0001-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:1:ost] [ 3398.339494] Lustre: lustre-OST0001-osc-MDT0000: update sequence from 0x100010000 to 0x280000400 [ 3408.357319] Lustre: lustre-MDT0000-lwp-OST0000: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 3408.363738] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 3408.369987] Lustre: Skipped 1 previous similar message [ 3411.133960] Lustre: server umount lustre-MDT0000 complete [ 3416.610649] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: (null) [ 3421.236746] LustreError: 117051:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1788642118 with bad export cookie 9312453292483475216 [ 3421.252647] LustreError: MGC192.168.203.150@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 3421.370962] Lustre: server umount lustre-OST0000 complete [ 3424.907681] Lustre: server umount lustre-OST0001 complete [ 3433.571462] Lustre: DEBUG MARKER: == conf-sanity test 88a: tunefs.lustre persists a --mountfsoptions-only change ========================================================== 17:02:09 (1788642129) [ 3458.393184] Lustre: DEBUG MARKER: == conf-sanity test 88b: check the default mount options can be overridden ========================================================== 17:02:33 (1788642153) [ 3470.194371] Lustre: DEBUG MARKER: oleg350-server.virtnet: executing load_modules_local [ 3493.568462] Lustre: DEBUG MARKER: == conf-sanity test 89: check tunefs --param and --erase-param{s} options ========================================================== 17:03:08 (1788642188) [ 3532.211453] Lustre: DEBUG MARKER: oleg350-server.virtnet: executing set_hostid [ 3542.203737] Lustre: DEBUG MARKER: oleg350-server.virtnet: executing load_modules_local [ 3578.488743] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 3578.777612] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 3578.807071] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 3578.938591] Lustre: lustre-MDT0000: new disk, initializing [ 3579.069259] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 3579.086728] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 3583.931200] Lustre: DEBUG MARKER: oleg350-server.virtnet: executing set_default_debug -1 all [ 3594.749742] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 3594.936299] Lustre: 126494:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 3595.004830] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [ 3595.011023] Lustre: Skipped 1 previous similar message [ 3595.157920] Lustre: lustre-MDT0001: new disk, initializing [ 3595.292546] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 3595.349133] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [ 3595.362278] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 3600.254255] Lustre: DEBUG MARKER: oleg350-server.virtnet: executing set_default_debug -1 all [ 3610.552066] Lustre: DEBUG MARKER: oleg350-client.virtnet: executing wait_import_state_mount FULL failover.node=192.0.2.254@tcp0,mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 3616.616385] Lustre: DEBUG MARKER: oleg350-client.virtnet: executing wait_import_state_mount FULL failover.node=192.0.2.254@tcp0,mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 3623.300470] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 3623.611294] Lustre: lustre-OST0000: new disk, initializing [ 3623.617748] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 3623.625101] Lustre: 128461:0:(osd_compat.c:1353:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 3623.775808] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 3630.415524] Lustre: DEBUG MARKER: oleg350-server.virtnet: executing set_default_debug -1 all [ 3631.658890] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [ 3631.676367] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 3631.802747] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 3640.156375] Lustre: DEBUG MARKER: oleg350-client.virtnet: executing wait_import_state_mount (FULL|IDLE) failover.node=192.0.2.254@tcp0,osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 3645.335674] Lustre: DEBUG MARKER: oleg350-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 [ 3645.581063] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [ 3650.525658] Lustre: DEBUG MARKER: oleg350-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid 50 [ 3650.741242] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [ 3652.580447] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 3652.582858] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 3652.611086] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 3652.629069] Lustre: Skipped 2 previous similar messages [ 3657.702080] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 3657.715958] Lustre: Skipped 1 previous similar message [ 3658.743662] Lustre: server umount lustre-OST0000 complete [ 3662.817739] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 3662.833043] Lustre: Skipped 1 previous similar message [ 3662.847853] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 3662.859116] Lustre: Skipped 1 previous similar message [ 3667.939704] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 3667.953051] Lustre: Skipped 1 previous similar message [ 3668.969449] Lustre: server umount lustre-MDT0000 complete [ 3672.862352] LustreError: 127288:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1788642369 with bad export cookie 9312453292483476441 [ 3672.864542] LustreError: MGC192.168.203.150@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 3672.886397] LustreError: 127288:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 6 previous similar messages [ 3673.292371] Lustre: server umount lustre-MDT0001 complete [ 3683.566033] Lustre: DEBUG MARKER: == conf-sanity test 90a: check max_mod_rpcs_in_flight is enforced ========================================================== 17:06:19 (1788642379) [ 3690.810584] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 3691.411906] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 3695.381729] Lustre: DEBUG MARKER: oleg350-server.virtnet: executing set_default_debug -1 all [ 3696.610242] LustreError: 130875:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 3696.620920] LustreError: 130875:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 2 previous similar messages [ 3701.730161] LustreError: 130874:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 3703.316616] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 3708.312497] Lustre: DEBUG MARKER: oleg350-server.virtnet: executing set_default_debug -1 all [ 3716.735578] Lustre: DEBUG MARKER: oleg350-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 3722.500635] Lustre: DEBUG MARKER: oleg350-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 3729.400226] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 3729.759643] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 3729.766610] Lustre: Skipped 1 previous similar message [ 3737.134482] Lustre: DEBUG MARKER: oleg350-server.virtnet: executing set_default_debug -1 all [ 3745.053699] Lustre: DEBUG MARKER: oleg350-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 3752.950108] Lustre: *** cfs_fail_loc=159, val=0*** [ 3752.957492] Lustre: Skipped 1 previous similar message [ 3768.465761] Lustre: lustre-MDT0000: Client 21009e02-9a12-404c-a315-3cf0db79d1ff (at 192.168.203.50@tcp) reconnecting [ 3770.001447] Lustre: *** cfs_fail_loc=159, val=0*** [ 3770.014414] Lustre: Skipped 5 previous similar messages [ 3786.416916] Lustre: lustre-MDT0000: Client 21009e02-9a12-404c-a315-3cf0db79d1ff (at 192.168.203.50@tcp) reconnecting [ 3792.806394] Lustre: server umount lustre-OST0000 complete [ 3799.521776] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 3799.534740] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 3799.537536] Lustre: Skipped 2 previous similar messages [ 3799.562063] Lustre: Skipped 1 previous similar message [ 3806.796505] Lustre: server umount lustre-MDT0000 complete [ 3809.761379] LustreError: 131602:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 3809.784637] LustreError: 131602:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 1 previous similar message [ 3810.484536] LustreError: 133597:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1788642507 with bad export cookie 9312453292483477323 [ 3810.492548] LustreError: MGC192.168.203.150@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 3810.500814] LustreError: 133597:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 3 previous similar messages [ 3821.197816] Lustre: server umount lustre-MDT0001 complete [ 3834.388303] Lustre: DEBUG MARKER: oleg350-server.virtnet: executing unload_modules_local [ 3837.096162] Key type lgssc unregistered [ 3837.436604] LNet: 134651:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 3837.444277] LNetError: 134651:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 3837.458832] LNet: Removed LNI 192.168.203.150@tcp [ 3838.262056] Key type .llcrypt unregistered [ 3838.266175] Key type ._llcrypt unregistered [ 3863.617395] Key type ._llcrypt registered [ 3863.622247] Key type .llcrypt registered [ 3863.754948] Lustre: DEBUG MARKER: oleg350-server.virtnet: executing load_modules_local [ 3864.860583] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 3865.006899] alg: No test for adler32 (adler32-zlib) [ 3866.144639] Lustre: Lustre: Build Version: 2.17.58_3_g980e1e2 [ 3866.380109] LNet: Added LNI 192.168.203.150@tcp [8/256/0/180] [ 3868.079168] Key type lgssc registered [ 3869.323795] Lustre: Echo OBD driver; http://www.lustre.org/ [ 3877.224387] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 3877.244061] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 3878.743285] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 3883.850053] Lustre: DEBUG MARKER: oleg350-server.virtnet: executing set_default_debug -1 all [ 3888.101614] LustreError: 136063:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 3888.127331] LustreError: 136063:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 2 previous similar messages [ 3893.218048] LustreError: 136062:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 3893.274375] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 3893.747496] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 3898.248261] Lustre: DEBUG MARKER: oleg350-server.virtnet: executing set_default_debug -1 all [ 3906.734418] Lustre: DEBUG MARKER: oleg350-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 3913.577854] Lustre: DEBUG MARKER: oleg350-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 3921.926820] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 3922.508885] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 3923.517093] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:42 to 0x280000401:65) [ 3930.266664] Lustre: DEBUG MARKER: oleg350-server.virtnet: executing set_default_debug -1 all [ 3940.578318] Lustre: DEBUG MARKER: oleg350-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 3951.091153] Lustre: *** cfs_fail_loc=159, val=0*** [ 3967.693139] Lustre: lustre-MDT0000: Client b8ba17d3-017d-4ecc-9507-da53245fa381 (at 192.168.203.50@tcp) reconnecting [ 3970.119334] Lustre: *** cfs_fail_loc=159, val=0*** [ 3986.621754] Lustre: lustre-MDT0001: Client b8ba17d3-017d-4ecc-9507-da53245fa381 (at 192.168.203.50@tcp) reconnecting [ 3988.677047] Lustre: *** cfs_fail_loc=159, val=0*** [ 3988.687217] Lustre: Skipped 5 previous similar messages [ 4005.077157] Lustre: lustre-MDT0001: Client b8ba17d3-017d-4ecc-9507-da53245fa381 (at 192.168.203.50@tcp) reconnecting [ 4010.912663] Lustre: 139099:0:(mdt_handler.c:64:mdt_max_mod_rpcs_per_client_set()) max_mod_rpcs_per_client is deprecated, set mdt.*.max_mod_rpcs_in_flight parameter instead [ 4013.952914] Lustre: *** cfs_fail_loc=159, val=0*** [ 4013.954918] Lustre: Skipped 2 previous similar messages [ 4029.593262] Lustre: lustre-MDT0000: Client 9ebaf000-1dda-4061-a33d-4e05dc95a470 (at 192.168.203.50@tcp) reconnecting [ 4031.582924] Lustre: *** cfs_fail_loc=159, val=0*** [ 4031.589528] Lustre: Skipped 16 previous similar messages [ 4048.056146] Lustre: lustre-MDT0000: Client 9ebaf000-1dda-4061-a33d-4e05dc95a470 (at 192.168.203.50@tcp) reconnecting [ 4049.519434] Lustre: 139397:0:(mdt_handler.c:64:mdt_max_mod_rpcs_per_client_set()) max_mod_rpcs_per_client is deprecated, set mdt.*.max_mod_rpcs_in_flight parameter instead [ 4056.036124] LustreError: lustre-OST0000-osc-MDT0001: operation ost_destroy to node 0@lo failed: rc = -107 [ 4056.036453] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4056.050178] LustreError: Skipped 2 previous similar messages [ 4056.069887] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 4056.548544] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4056.581426] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 4060.716261] Lustre: server umount lustre-OST0000 complete [ 4066.785931] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4066.791042] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 4066.802601] Lustre: Skipped 1 previous similar message [ 4066.819792] Lustre: Skipped 1 previous similar message [ 4071.032282] Lustre: server umount lustre-MDT0000 complete [ 4071.905631] LustreError: 139161:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 4071.922851] LustreError: 139161:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 1 previous similar message [ 4074.354961] LustreError: 136043:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1788642771 with bad export cookie 12799612098817349375 [ 4074.364673] LustreError: MGC192.168.203.150@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 4074.367409] LustreError: 136043:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 2 previous similar messages [ 4074.809931] Lustre: server umount lustre-MDT0001 complete [ 4085.284257] Lustre: DEBUG MARKER: oleg350-server.virtnet: executing unload_modules_local [ 4088.124471] Key type lgssc unregistered [ 4088.367088] LNet: 140448:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 4088.379484] LNetError: 140448:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 4088.394473] LNet: Removed LNI 192.168.203.150@tcp [ 4089.220450] Key type .llcrypt unregistered [ 4089.226397] Key type ._llcrypt unregistered [ 4113.370519] Key type ._llcrypt registered [ 4113.372424] Key type .llcrypt registered [ 4113.460890] Lustre: DEBUG MARKER: oleg350-server.virtnet: executing load_modules_local [ 4114.382742] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 4114.456230] alg: No test for adler32 (adler32-zlib) [ 4115.518699] Lustre: Lustre: Build Version: 2.17.58_3_g980e1e2 [ 4115.726860] LNet: Added LNI 192.168.203.150@tcp [8/256/0/180] [ 4117.383564] Key type lgssc registered [ 4118.420816] Lustre: Echo OBD driver; http://www.lustre.org/ [ 4125.924161] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 4125.958465] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 4127.348481] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 4131.731809] Lustre: DEBUG MARKER: oleg350-server.virtnet: executing set_default_debug -1 all [ 4137.960477] LustreError: 141855:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 4137.985651] LustreError: 141855:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 2 previous similar messages [ 4140.534883] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 4140.829558] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 4144.980285] Lustre: DEBUG MARKER: oleg350-server.virtnet: executing set_default_debug -1 all [ 4152.082407] Lustre: DEBUG MARKER: oleg350-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 4157.164740] Lustre: DEBUG MARKER: oleg350-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 4164.332367] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 4164.734289] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 4165.760104] Lustre: lustre-OST0000: new connection from lustre-MDT0001-mdtlov (cleaning up unused objects from 0x280000400:72 to 0x280000400:97) [ 4171.193464] Lustre: DEBUG MARKER: oleg350-server.virtnet: executing set_default_debug -1 all [ 4172.955112] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:85 to 0x280000401:129) [ 4178.903800] Lustre: DEBUG MARKER: oleg350-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 4185.034336] Lustre: 144438:0:(mdt_handler.c:64:mdt_max_mod_rpcs_per_client_set()) max_mod_rpcs_per_client is deprecated, set mdt.*.max_mod_rpcs_in_flight parameter instead [ 4187.974261] Lustre: 144487:0:(mdt_handler.c:64:mdt_max_mod_rpcs_per_client_set()) max_mod_rpcs_per_client is deprecated, set mdt.*.max_mod_rpcs_in_flight parameter instead [ 4189.320088] Lustre: Modifying parameter lustre.mdc.lustre-MDT0000-mdc-*.max_rpcs_in_flight=8 in log params [ 4190.230433] Lustre: Setting parameter lustre.mdc.lustre-MDT0000-mdc-*.max_mod_rpcs_in_flight=8 in log params [ 4194.570916] Lustre: 144634:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre/mdc.lustre-MDT0000-mdc-*.max_rpcs_in_flight=8 (mode = 0) failed: rc = -17 [ 4195.539577] Lustre: Modifying parameter lustre.mdc.lustre-MDT0000-mdc-*.max_mod_rpcs_in_flight=7 in log params [ 4196.557974] Lustre: 144732:0:(mdt_handler.c:64:mdt_max_mod_rpcs_per_client_set()) max_mod_rpcs_per_client is deprecated, set mdt.*.max_mod_rpcs_in_flight parameter instead [ 4201.961601] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 4201.974552] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4202.012682] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 4204.011174] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4204.012818] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 4204.036579] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 4204.046753] Lustre: Skipped 1 previous similar message [ 4207.851758] Lustre: server umount lustre-OST0000 complete [ 4212.710937] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 4212.723911] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4212.744092] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 4218.240623] Lustre: server umount lustre-MDT0000 complete [ 4219.375609] LustreError: 141855:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 4221.977519] LustreError: 142589:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1788642918 with bad export cookie 9814848666547984429 [ 4221.984500] LustreError: MGC192.168.203.150@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 4221.992611] LustreError: 142589:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 4222.355418] Lustre: server umount lustre-MDT0001 complete [ 4233.048554] Lustre: DEBUG MARKER: oleg350-server.virtnet: executing unload_modules_local [ 4236.044142] Key type lgssc unregistered [ 4236.366603] LNet: 145793:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 4236.376079] LNetError: 145793:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 4236.395493] LNet: Removed LNI 192.168.203.150@tcp [ 4237.296263] Key type .llcrypt unregistered [ 4237.298385] Key type ._llcrypt unregistered [ 4263.570240] Key type ._llcrypt registered [ 4263.572122] Key type .llcrypt registered [ 4263.655600] Lustre: DEBUG MARKER: oleg350-server.virtnet: executing load_modules_local [ 4264.922972] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 4264.956054] alg: No test for adler32 (adler32-zlib) [ 4266.030646] Lustre: Lustre: Build Version: 2.17.58_3_g980e1e2 [ 4266.322274] LNet: Added LNI 192.168.203.150@tcp [8/256/0/180] [ 4267.999167] Key type lgssc registered [ 4269.175154] Lustre: Echo OBD driver; http://www.lustre.org/ [ 4277.715543] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 4277.736319] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 4279.296414] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 4284.473985] Lustre: DEBUG MARKER: oleg350-server.virtnet: executing set_default_debug -1 all [ 4289.506289] LustreError: 147204:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 4289.533073] LustreError: 147204:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 2 previous similar messages [ 4293.901241] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 4294.325370] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 4299.297804] Lustre: DEBUG MARKER: oleg350-server.virtnet: executing set_default_debug -1 all [ 4308.507513] Lustre: DEBUG MARKER: oleg350-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 4315.511305] Lustre: DEBUG MARKER: oleg350-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 4324.146408] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 4324.322360] LustreError: 149032:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-OST0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 4324.590474] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 4329.980155] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:85 to 0x280000401:161) [ 4331.399310] Lustre: DEBUG MARKER: oleg350-server.virtnet: executing set_default_debug -1 all [ 4339.201074] Lustre: lustre-OST0000: new connection from lustre-MDT0001-mdtlov (cleaning up unused objects from 0x280000400:72 to 0x280000400:129) [ 4340.174202] Lustre: DEBUG MARKER: oleg350-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 4347.613737] Lustre: *** cfs_fail_loc=159, val=0*** [ 4347.622355] Lustre: Skipped 1 previous similar message [ 4363.952773] Lustre: lustre-MDT0000: Client 48c3e9f3-638a-4bac-920d-b3749d37990e (at 192.168.203.50@tcp) reconnecting [ 4369.889878] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4369.897229] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 4369.903645] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 4375.009620] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 4375.019338] Lustre: Skipped 1 previous similar message [ 4375.457692] Lustre: server umount lustre-OST0000 complete [ 4380.128540] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4380.157493] Lustre: Skipped 1 previous similar message [ 4380.169662] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 4380.185617] Lustre: Skipped 2 previous similar messages [ 4385.250856] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 4385.264682] Lustre: Skipped 1 previous similar message [ 4385.992851] Lustre: server umount lustre-MDT0000 complete [ 4390.327191] LustreError: 147945:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1788643087 with bad export cookie 13256605237446665876 [ 4390.333608] LustreError: MGC192.168.203.150@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 4390.341665] LustreError: 147945:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 4391.063502] Lustre: server umount lustre-MDT0001 complete [ 4404.833785] Lustre: DEBUG MARKER: oleg350-server.virtnet: executing unload_modules_local [ 4408.224742] Key type lgssc unregistered [ 4408.600689] LNet: 150872:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 4408.623629] LNetError: 150872:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 4408.642449] LNet: Removed LNI 192.168.203.150@tcp [ 4409.722582] Key type .llcrypt unregistered [ 4409.726915] Key type ._llcrypt unregistered [ 4439.673629] Key type ._llcrypt registered [ 4439.678672] Key type .llcrypt registered [ 4439.808314] Lustre: DEBUG MARKER: oleg350-server.virtnet: executing load_modules_local [ 4441.046032] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 4441.144631] alg: No test for adler32 (adler32-zlib) [ 4442.211453] Lustre: Lustre: Build Version: 2.17.58_3_g980e1e2 [ 4442.559354] LNet: Added LNI 192.168.203.150@tcp [8/256/0/180] [ 4444.287229] Key type lgssc registered [ 4445.671208] Lustre: Echo OBD driver; http://www.lustre.org/ [ 4454.323369] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 4454.361767] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 4455.858191] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 4460.536652] Lustre: DEBUG MARKER: oleg350-server.virtnet: executing set_default_debug -1 all [ 4465.134451] LustreError: 152283:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 4465.151579] LustreError: 152283:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 2 previous similar messages [ 4469.321184] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 4469.706693] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 4474.152484] Lustre: DEBUG MARKER: oleg350-server.virtnet: executing set_default_debug -1 all [ 4482.559676] Lustre: DEBUG MARKER: oleg350-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 4488.395699] Lustre: DEBUG MARKER: oleg350-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 4495.755923] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 4496.129776] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 4500.251893] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:170 to 0x280000401:193) [ 4502.395835] Lustre: DEBUG MARKER: oleg350-server.virtnet: executing set_default_debug -1 all [ 4509.260149] Lustre: DEBUG MARKER: oleg350-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 4514.560897] Lustre: lustre-OST0000: new connection from lustre-MDT0001-mdtlov (cleaning up unused objects from 0x280000400:72 to 0x280000400:161) [ 4534.476034] Lustre: lustre-MDT0001: evicting d11d70be-1adb-434c-b061-1dcc96cc8068 (at 192.168.203.50@tcp) by administrative request [ 4534.490686] Lustre: Skipped 1 previous similar message [ 4565.984935] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4565.987578] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 4565.992666] Lustre: Skipped 1 previous similar message [ 4566.006818] Lustre: Skipped 1 previous similar message [ 4569.517448] Lustre: server umount lustre-OST0000 complete [ 4576.229339] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4576.240406] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 4576.247258] Lustre: Skipped 1 previous similar message [ 4576.257585] Lustre: Skipped 1 previous similar message [ 4579.685298] Lustre: server umount lustre-MDT0000 complete [ 4581.348710] LustreError: 152283:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 4581.385876] LustreError: 152283:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 1 previous similar message [ 4583.664512] LustreError: 152265:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1788643280 with bad export cookie 4572872034712433654 [ 4583.680038] LustreError: 152265:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 1 previous similar message [ 4583.680631] LustreError: MGC192.168.203.150@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 4584.066442] Lustre: server umount lustre-MDT0001 complete [ 4596.949254] Lustre: DEBUG MARKER: oleg350-server.virtnet: executing unload_modules_local [ 4599.740956] Key type lgssc unregistered [ 4600.025794] LNet: 156352:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 4600.032208] LNetError: 156352:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 4600.045785] LNet: Removed LNI 192.168.203.150@tcp [ 4600.937128] Key type .llcrypt unregistered [ 4600.939656] Key type ._llcrypt unregistered [ 4672.978355] Key type ._llcrypt registered [ 4672.980726] Key type .llcrypt registered [ 4673.066669] Lustre: DEBUG MARKER: oleg350-server.virtnet: executing load_modules_local [ 4674.095908] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 4674.232645] alg: No test for adler32 (adler32-zlib) [ 4675.392945] Lustre: Lustre: Build Version: 2.17.58_3_g980e1e2 [ 4675.662894] LNet: Added LNI 192.168.203.150@tcp [8/256/0/180] [ 4677.335191] Key type lgssc registered [ 4678.645229] Lustre: Echo OBD driver; http://www.lustre.org/ [ 4686.629565] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 4686.654137] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 4688.157048] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 4692.748307] Lustre: DEBUG MARKER: oleg350-server.virtnet: executing set_default_debug -1 all [ 4698.598548] LustreError: 158842:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 4698.629117] LustreError: 158842:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 2 previous similar messages [ 4701.197727] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 4701.674662] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 4705.618747] Lustre: DEBUG MARKER: oleg350-server.virtnet: executing set_default_debug -1 all [ 4713.201778] Lustre: DEBUG MARKER: oleg350-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 4719.349407] Lustre: DEBUG MARKER: oleg350-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 4726.438817] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 4726.755978] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 4732.825849] Lustre: DEBUG MARKER: oleg350-server.virtnet: executing set_default_debug -1 all [ 4733.966590] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:195 to 0x280000401:225) [ 4740.610592] Lustre: DEBUG MARKER: oleg350-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 4746.283981] Lustre: lustre-OST0000: new connection from lustre-MDT0001-mdtlov (cleaning up unused objects from 0x280000400:72 to 0x280000400:193) [ 4754.401616] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 4754.405318] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4754.414546] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 4756.453940] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 4756.454512] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4756.481237] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 4760.926638] Lustre: server umount lustre-OST0000 complete [ 4766.692659] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4766.713268] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 4766.716057] Lustre: Skipped 1 previous similar message [ 4771.560062] Lustre: server umount lustre-MDT0000 complete [ 4771.808711] LustreError: 159896:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 4771.848533] LustreError: 159896:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 1 previous similar message [ 4775.652091] LustreError: 158821:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1788643472 with bad export cookie 15418259728019400791 [ 4775.675127] LustreError: MGC192.168.203.150@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 4775.974415] Lustre: server umount lustre-MDT0001 complete [ 4786.979896] Lustre: DEBUG MARKER: oleg350-server.virtnet: executing unload_modules_local [ 4789.614631] Key type lgssc unregistered [ 4789.844845] LNet: 162400:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 4789.856554] LNetError: 162400:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 4789.880493] LNet: Removed LNI 192.168.203.150@tcp [ 4790.587791] Key type .llcrypt unregistered [ 4790.590303] Key type ._llcrypt unregistered [ 4820.548432] Key type ._llcrypt registered [ 4820.550973] Key type .llcrypt registered [ 4820.636750] Lustre: DEBUG MARKER: oleg350-server.virtnet: executing load_modules_local [ 4821.299053] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 4821.369617] alg: No test for adler32 (adler32-zlib) [ 4822.447355] Lustre: Lustre: Build Version: 2.17.58_3_g980e1e2 [ 4822.784783] LNet: Added LNI 192.168.203.150@tcp [8/256/0/180] [ 4824.591194] Key type lgssc registered [ 4825.916433] Lustre: Echo OBD driver; http://www.lustre.org/ [ 4842.118421] Lustre: DEBUG MARKER: == conf-sanity test 100: check lshowmount lists MGS, MDT, OST and 0@lo ========================================================== 17:25:37 (1788643537) [ 4858.799830] Lustre: DEBUG MARKER: oleg350-server.virtnet: executing set_hostid [ 4870.635907] Lustre: DEBUG MARKER: oleg350-server.virtnet: executing load_modules_local [ 4908.949829] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 4908.968600] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 4910.230906] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 4910.262393] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 4910.364755] Lustre: lustre-MDT0000: new disk, initializing [ 4910.457691] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 4910.475952] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 4915.160378] Lustre: DEBUG MARKER: oleg350-server.virtnet: executing set_default_debug -1 all [ 4926.008031] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 4926.170349] Lustre: 166663:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 4926.257286] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [ 4926.260783] Lustre: Skipped 1 previous similar message [ 4926.358739] Lustre: lustre-MDT0001: new disk, initializing [ 4926.480722] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 4926.555458] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [ 4926.573315] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 4930.945851] Lustre: DEBUG MARKER: oleg350-server.virtnet: executing set_default_debug -1 all [ 4940.471703] Lustre: DEBUG MARKER: oleg350-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 4946.116686] Lustre: DEBUG MARKER: oleg350-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 4952.950044] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 4953.274188] Lustre: lustre-OST0000: new disk, initializing [ 4953.281147] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 4953.292654] Lustre: 168629:0:(osd_compat.c:1353:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 4953.419584] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 4959.868744] Lustre: DEBUG MARKER: oleg350-server.virtnet: executing set_default_debug -1 all [ 4960.835228] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [ 4960.845845] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 4960.986498] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 4970.130508] Lustre: DEBUG MARKER: oleg350-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 4981.746292] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 4981.749163] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4981.765144] LustreError: Skipped 1 previous similar message [ 4981.768755] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 4981.781734] Lustre: Skipped 1 previous similar message [ 4985.780840] Lustre: server umount lustre-OST0000 complete [ 4991.969368] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4991.974143] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 4991.989053] Lustre: Skipped 1 previous similar message [ 4991.992527] Lustre: Skipped 1 previous similar message [ 4995.631897] Lustre: server umount lustre-MDT0000 complete [ 4999.003633] LustreError: 166653:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1788643695 with bad export cookie 12478342815948524430 [ 4999.010713] LustreError: MGC192.168.203.150@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 4999.025623] LustreError: 166653:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 2 previous similar messages [ 4999.314450] Lustre: server umount lustre-MDT0001 complete [ 5010.821535] Lustre: DEBUG MARKER: oleg350-server.virtnet: executing unload_modules_local [ 5013.654925] Key type lgssc unregistered [ 5013.969533] LNet: 170534:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 5013.981901] LNetError: 170534:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 5014.014915] LNet: Removed LNI 192.168.203.150@tcp [ 5015.015745] Key type .llcrypt unregistered [ 5015.018892] Key type ._llcrypt unregistered [ 5037.675295] Key type ._llcrypt registered [ 5037.679382] Key type .llcrypt registered [ 5037.788161] Lustre: DEBUG MARKER: oleg350-server.virtnet: executing load_modules_local [ 5038.549199] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 5038.619306] alg: No test for adler32 (adler32-zlib) [ 5039.639399] Lustre: Lustre: Build Version: 2.17.58_3_g980e1e2 [ 5039.895939] LNet: Added LNI 192.168.203.150@tcp [8/256/0/180] [ 5041.647238] Key type lgssc registered [ 5042.637572] Lustre: Echo OBD driver; http://www.lustre.org/ [ 5051.179415] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 5051.223067] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 5052.794133] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 5057.041823] Lustre: DEBUG MARKER: oleg350-server.virtnet: executing set_default_debug -1 all [ 5062.120677] LustreError: 171957:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 5062.159149] LustreError: 171957:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 2 previous similar messages [ 5067.234928] LustreError: 171958:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 5067.650041] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 5068.092962] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 5072.310555] Lustre: DEBUG MARKER: oleg350-server.virtnet: executing set_default_debug -1 all [ 5081.682172] Lustre: DEBUG MARKER: oleg350-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 5088.823279] Lustre: DEBUG MARKER: oleg350-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 5096.265199] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 5096.635065] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 5097.673683] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:3 to 0x280000401:33) [ 5103.245598] Lustre: DEBUG MARKER: oleg350-server.virtnet: executing set_default_debug -1 all [ 5111.670317] Lustre: DEBUG MARKER: oleg350-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 5118.519454] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 5118.539595] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5118.554707] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 5118.562667] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5118.576177] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 5119.523861] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 5119.547799] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5119.569247] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 5119.582491] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5119.592623] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 5120.555560] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 5120.571465] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5120.582077] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 5120.587532] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5120.595310] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 5121.450865] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5122.387727] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5123.423313] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 5123.429592] Lustre: Skipped 2 previous similar messages [ 5123.452383] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5123.462024] Lustre: Skipped 2 previous similar messages [ 5123.473857] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 5123.480929] Lustre: Skipped 2 previous similar messages [ 5123.484755] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5123.504993] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 5123.514791] Lustre: Skipped 2 previous similar messages [ 5124.489159] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5125.402454] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5126.222137] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5127.149746] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5128.123693] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 5128.135768] Lustre: Skipped 4 previous similar messages [ 5128.151166] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5128.173840] Lustre: Skipped 4 previous similar messages [ 5128.179842] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 5128.186066] Lustre: Skipped 4 previous similar messages [ 5128.194061] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5128.210397] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 5128.220023] Lustre: Skipped 4 previous similar messages [ 5129.081996] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5130.027632] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5130.902361] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5131.817071] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5132.698985] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5133.486890] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5134.291230] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5135.265094] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5136.231837] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 5136.240722] Lustre: Skipped 8 previous similar messages [ 5136.265927] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5136.273515] Lustre: Skipped 8 previous similar messages [ 5136.277185] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 5136.280699] Lustre: Skipped 8 previous similar messages [ 5136.287347] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5136.313866] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 5136.323422] Lustre: Skipped 8 previous similar messages [ 5137.142634] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5137.945709] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5138.699542] LustreError: 171977:0:(osp_precreate.c:655:osp_precreate_send()) lustre-OST0000-osc-MDT0000: can't precreate: rc = -5 [ 5138.717609] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5138.722980] LustreError: 171977:0:(osp_precreate.c:1359:osp_precreate_thread()) lustre-OST0000-osc-MDT0000: cannot precreate objects: rc = -5 [ 5139.610381] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5140.537988] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5141.393342] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5142.196629] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5143.053285] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5143.789845] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5144.750171] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5145.803070] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5146.602311] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5147.667294] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5148.617797] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5149.622648] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5150.647420] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5151.575085] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5152.664874] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 5152.670097] Lustre: Skipped 17 previous similar messages [ 5152.680990] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5152.691192] Lustre: Skipped 17 previous similar messages [ 5152.699299] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 5152.703943] Lustre: Skipped 17 previous similar messages [ 5152.707764] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5152.716455] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 5152.719863] Lustre: Skipped 17 previous similar messages [ 5153.551057] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5154.589840] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5155.352499] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5156.015324] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5156.760601] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5157.710945] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5158.678993] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5159.873898] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5160.939620] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5161.819729] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5162.838878] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5164.028424] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5165.173884] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5166.158953] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5167.170360] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5168.095798] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5169.208943] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5170.177722] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5171.492685] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5172.622572] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5173.664032] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5174.917376] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5176.205144] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5177.388268] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5178.314256] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5179.397050] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5180.307294] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5181.340069] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5182.138744] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5183.181322] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5184.044469] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5185.137878] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 5185.150350] Lustre: Skipped 31 previous similar messages [ 5185.172375] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5185.188719] Lustre: Skipped 31 previous similar messages [ 5185.199691] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 5185.206067] Lustre: Skipped 31 previous similar messages [ 5185.214403] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5185.226600] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 5185.236482] Lustre: Skipped 31 previous similar messages [ 5186.296896] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5187.391599] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5188.597061] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5189.780645] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5190.729727] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5191.742729] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5192.787848] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5193.655709] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5194.781666] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5195.680995] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5196.622261] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5197.559241] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5198.546165] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5199.555280] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5200.659364] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5201.450062] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5202.408966] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5203.466834] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5204.495615] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5205.390400] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5206.433979] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5207.539792] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5208.757651] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5209.824910] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5211.025966] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5211.963073] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5212.963421] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5213.923084] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5214.894415] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5215.820344] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5216.831351] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5217.833706] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5218.753677] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5219.836180] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5220.838572] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5221.934775] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5223.109540] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5223.984120] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5224.940302] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5225.940741] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5226.956531] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5227.766123] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5228.560117] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5229.442268] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5230.501884] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5231.362406] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5232.332023] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5233.183946] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5234.076494] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5234.843732] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5235.932677] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5236.867147] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5237.933335] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5238.891833] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5239.895957] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5240.968270] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5241.975199] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5242.802118] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5243.574804] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5244.491700] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5245.481500] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5246.507365] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5247.299821] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5248.254281] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5249.175827] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 5249.185819] Lustre: Skipped 64 previous similar messages [ 5249.197295] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5249.208722] Lustre: Skipped 64 previous similar messages [ 5249.214813] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 5249.222021] Lustre: Skipped 64 previous similar messages [ 5249.225964] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5249.248195] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 5249.263076] Lustre: Skipped 64 previous similar messages [ 5250.247917] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5251.195374] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5252.211303] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5253.236511] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5254.126665] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5254.994085] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5255.825234] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5256.855573] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5257.962322] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5259.121486] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5260.524357] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5261.499213] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5262.644214] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5263.480667] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5264.368551] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5265.277550] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5266.405870] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5267.343268] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5268.573335] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5269.747204] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5270.985101] LustreError: 171977:0:(osp_precreate.c:655:osp_precreate_send()) lustre-OST0000-osc-MDT0000: can't precreate: rc = -5 [ 5271.004921] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5271.006480] LustreError: 171977:0:(osp_precreate.c:1359:osp_precreate_thread()) lustre-OST0000-osc-MDT0000: cannot precreate objects: rc = -5 [ 5271.868590] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5272.695223] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5273.701654] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5274.545938] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5275.461237] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5276.542750] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5277.802986] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5278.783735] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5279.854946] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5280.696852] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5281.973722] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5283.273879] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5284.224116] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5285.214599] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5286.129778] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5287.299814] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5288.214060] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5289.141542] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5290.111189] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5291.054704] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5292.012794] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5293.192285] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5294.028147] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5294.974128] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5295.938806] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5296.886231] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5297.991995] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5299.013863] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5300.200705] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5301.174677] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5302.333687] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5303.276823] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5304.247060] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5305.093627] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5306.118080] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5307.207617] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5308.366990] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5309.555219] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5310.646598] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5311.586306] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5312.683512] LustreError: 171977:0:(osp_precreate.c:655:osp_precreate_send()) lustre-OST0000-osc-MDT0000: can't precreate: rc = -5 [ 5312.695543] LustreError: 171977:0:(osp_precreate.c:1359:osp_precreate_thread()) lustre-OST0000-osc-MDT0000: cannot precreate objects: rc = -5 [ 5312.724742] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5313.560595] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5314.580098] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5315.639987] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5316.589228] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5317.549713] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5318.607103] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5319.317956] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5320.227443] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5321.054961] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5322.037369] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5322.680474] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5323.559392] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5324.445504] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5325.461946] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5326.313662] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5327.119772] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5328.080365] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5329.057729] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5329.881120] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5330.882340] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5331.896649] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5332.653121] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5333.620528] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5334.398479] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5335.573179] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5336.780730] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5337.998777] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5339.117915] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5340.342487] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5341.388585] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5342.341608] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5343.313317] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5344.394566] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5345.503506] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5346.637332] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5347.715632] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5348.859063] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5349.863550] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5350.991207] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5352.037524] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5352.990268] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5354.022239] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5355.018651] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5356.045626] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5356.997384] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5358.028957] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5359.007554] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5360.066033] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5361.088150] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5362.311407] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5363.335841] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5364.347958] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5365.414396] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5366.469917] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5367.626520] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5368.710339] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5369.780777] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5370.729817] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5371.655790] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5372.489487] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5373.645316] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5374.845428] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5376.117475] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5377.191403] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5378.122442] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 5378.130559] Lustre: Skipped 126 previous similar messages [ 5378.149765] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5378.162939] Lustre: Skipped 126 previous similar messages [ 5378.167761] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 5378.175830] Lustre: Skipped 126 previous similar messages [ 5378.182939] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5378.208124] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 5378.221811] Lustre: Skipped 126 previous similar messages [ 5379.211359] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5380.267422] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5381.221430] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5382.250119] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5383.216273] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5384.416545] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5385.642396] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5386.708853] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5387.980437] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5389.019452] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5389.978591] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5391.110022] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5391.974989] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5392.884215] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5393.770955] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5394.667727] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5395.507121] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5396.363624] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5397.322058] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5398.393535] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5399.217715] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5400.182237] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5401.159738] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5402.278655] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5403.521752] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5404.478208] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5405.787711] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5406.820471] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5408.042613] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5408.973601] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5410.043275] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5410.968971] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5412.018875] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5413.006063] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5413.959452] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5414.990823] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5415.983597] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5417.156935] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5418.064318] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5419.089815] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5420.116164] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5421.198586] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5422.252564] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5423.218821] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5424.244076] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5425.404335] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5426.345927] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5427.353553] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5428.284809] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5429.315484] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5430.233149] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5431.254469] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5432.103869] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5433.198574] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5434.260146] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5435.383414] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5436.415269] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5437.385922] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5438.358385] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5439.450421] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5440.475648] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5441.768204] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5442.785595] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5443.791817] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5445.028457] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5446.073279] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5446.868795] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5447.767942] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5448.718986] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5449.484166] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5450.405931] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5451.307213] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5452.166816] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5453.191704] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5454.210064] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5455.484986] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5456.424091] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5457.268314] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5458.282390] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5459.250577] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5460.152454] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5460.993484] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5461.933615] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5462.904956] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5463.854206] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5464.931315] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5465.871704] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5466.903573] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5467.907949] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5468.971912] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5469.936793] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5470.948913] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5471.860893] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5472.895994] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5473.822879] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5474.847202] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5475.888511] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5476.864810] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5478.076066] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5479.231868] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5480.364480] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5481.408660] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5482.496400] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5483.685761] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5484.932753] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5486.104854] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5486.915217] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5487.952371] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5489.084814] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5490.189255] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5491.386432] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5492.582321] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5493.740884] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5494.706236] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5495.861288] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5497.057548] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5498.274476] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5499.327020] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5500.273061] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5501.616506] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5503.099747] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5504.212920] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5505.481370] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5506.641640] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5507.846401] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5509.047443] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5510.200112] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5511.107594] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5512.052041] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5512.903078] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5513.947800] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5514.634134] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5515.589305] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5516.553062] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5517.487734] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5518.225469] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5518.964976] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5519.771860] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5520.801770] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5521.549449] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5522.652252] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5523.699437] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5524.754611] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5525.902831] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5527.194798] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5528.351573] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5529.533076] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5530.714488] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5531.870093] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5533.103747] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5534.030168] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5535.030536] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5536.076472] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5537.059398] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5538.183420] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5539.273832] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5540.247730] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5541.185728] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5542.479704] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5543.783135] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5545.065994] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5546.190162] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5547.451085] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5548.544696] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5549.285669] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5549.976685] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5550.604922] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5551.758934] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5552.914993] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5553.850278] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5554.935440] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5555.654886] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5556.680176] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5557.605211] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5558.976739] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5560.000782] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5560.886281] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5562.156067] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5563.968503] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5565.530984] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5566.802156] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5568.374556] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5569.854416] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5571.204232] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5573.010716] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5574.107111] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5575.256757] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5576.684300] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5578.291734] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5579.392835] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5580.586876] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5581.622072] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5582.497087] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5583.283605] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5584.198866] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5585.021963] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5586.062771] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5587.050718] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5587.924745] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5589.014299] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5589.829599] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5590.822731] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5591.769854] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5592.608821] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5593.638315] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5594.703663] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5595.748306] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5596.733208] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5597.921763] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5598.876416] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5599.993710] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5601.087350] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5602.103256] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5603.242731] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5604.197823] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5605.167782] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5606.114620] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5607.173564] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5608.151365] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5609.057032] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5609.829305] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5610.707418] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5611.534436] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5612.654116] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5614.141940] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5615.550319] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5616.990608] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5617.869443] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5619.253404] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5620.221786] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5621.287362] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5622.427324] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5623.512679] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5624.953549] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5626.388535] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5627.967955] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5629.580713] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5631.129219] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5632.461025] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5633.888359] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5635.417165] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 5635.424319] Lustre: Skipped 240 previous similar messages [ 5635.429327] LustreError: 171977:0:(osp_precreate.c:655:osp_precreate_send()) lustre-OST0000-osc-MDT0000: can't precreate: rc = -5 [ 5635.439364] LustreError: 171977:0:(osp_precreate.c:1359:osp_precreate_thread()) lustre-OST0000-osc-MDT0000: cannot precreate objects: rc = -5 [ 5635.447508] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5635.447516] Lustre: Skipped 240 previous similar messages [ 5635.449254] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 5635.472540] Lustre: Skipped 240 previous similar messages [ 5635.475354] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5635.492228] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 5635.502147] Lustre: Skipped 240 previous similar messages [ 5636.576701] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5637.382545] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5638.763685] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5639.874249] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5640.849866] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5641.949271] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5642.997495] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5644.029050] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5644.933528] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5645.974207] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5647.006974] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5647.882756] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5649.050443] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5650.298349] LustreError: 171977:0:(osp_precreate.c:655:osp_precreate_send()) lustre-OST0000-osc-MDT0000: can't precreate: rc = -5 [ 5650.317533] LustreError: 171977:0:(osp_precreate.c:1359:osp_precreate_thread()) lustre-OST0000-osc-MDT0000: cannot precreate objects: rc = -5 [ 5650.330526] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5651.486424] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5652.865412] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5654.322276] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5655.195679] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5656.418799] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5657.932822] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5658.695826] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5659.583235] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5660.885602] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5662.229390] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5663.609147] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5664.895318] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5666.172907] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5667.490070] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5668.710254] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5670.190423] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5671.736435] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5672.927267] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5673.874832] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5675.117052] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5676.304633] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5677.380217] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5678.367470] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5679.611493] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5680.524212] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5681.737256] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5682.669858] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5684.215341] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5685.492645] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5686.602087] LustreError: 171977:0:(osp_precreate.c:655:osp_precreate_send()) lustre-OST0000-osc-MDT0000: can't precreate: rc = -5 [ 5686.615670] LustreError: 171977:0:(osp_precreate.c:1359:osp_precreate_thread()) lustre-OST0000-osc-MDT0000: cannot precreate objects: rc = -5 [ 5686.617720] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5687.846543] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5689.034984] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5690.022951] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5691.073262] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5691.899977] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5692.669706] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5693.678589] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5694.942891] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5696.078758] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5697.036566] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5698.213671] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5699.276072] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5700.339891] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5701.242388] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5702.395154] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5703.301740] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5704.558787] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5705.549043] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5706.491139] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5707.300791] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5708.234133] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5709.138244] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5710.205196] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5711.199094] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5712.181956] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5713.151130] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5714.218548] LustreError: 171977:0:(osp_precreate.c:655:osp_precreate_send()) lustre-OST0000-osc-MDT0000: can't precreate: rc = -5 [ 5714.234050] LustreError: 171977:0:(osp_precreate.c:1359:osp_precreate_thread()) lustre-OST0000-osc-MDT0000: cannot precreate objects: rc = -5 [ 5714.244894] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5715.235781] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5716.173737] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5717.142041] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5718.182237] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5719.094440] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5719.972252] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5721.022395] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5721.822263] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5722.596997] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5723.521161] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5724.684067] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5725.736399] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5726.717096] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5727.756976] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5728.791554] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5729.849737] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5730.932295] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5731.868194] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5732.793007] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5733.822697] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5734.626263] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5735.476201] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5736.357969] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5737.318547] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5738.308635] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5739.296029] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5740.398881] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5741.158726] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5742.083803] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5742.961097] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5743.830159] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5744.759591] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5745.652819] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5746.547468] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5747.597320] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5748.537570] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5749.497437] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5750.327292] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5751.373491] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5752.392428] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5753.441495] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5754.261282] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5755.247275] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5756.156356] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5757.070532] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5758.062727] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5759.099332] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5760.206016] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5761.340566] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5762.485197] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5763.648938] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5764.570550] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5765.558054] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5766.446962] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5767.546479] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5768.580717] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5769.475102] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5770.377380] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5771.158957] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5772.011100] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5773.052967] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5774.023032] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5775.051428] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5776.002796] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5777.116733] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5778.090655] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5779.224529] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5780.137489] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5780.920440] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5781.708935] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5782.670844] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5783.399720] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5784.299492] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5785.069638] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5785.901891] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5786.874538] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5787.791711] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5788.779641] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5789.743791] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5790.729776] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5791.675841] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5792.815493] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5793.671728] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5794.676487] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5795.586171] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5796.547166] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5797.577750] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5798.422655] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5799.347188] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5800.301320] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5801.250837] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5802.119504] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5802.977567] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5803.834417] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5804.716341] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5805.511850] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5806.523943] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5807.361641] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5808.292065] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5809.137768] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5810.066643] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5811.007361] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5811.869902] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5812.895648] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5813.869148] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5814.680772] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5815.808070] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5816.683055] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5817.842833] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5819.043453] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5820.089966] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5821.154226] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5822.255206] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5823.641397] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5824.480932] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5825.332734] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5826.426229] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5827.445522] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5828.645692] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5829.779724] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5830.980510] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5831.878805] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5832.878168] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5833.889335] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5835.060490] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5835.778698] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5836.960899] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5837.905996] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5838.891775] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5839.748353] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5840.844460] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5841.923967] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5843.160958] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5844.289175] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5845.171724] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5846.039645] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5846.864262] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5847.711650] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5848.519217] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5849.625575] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5850.740105] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5851.694742] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5852.689104] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5853.862436] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5855.055235] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5856.401345] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5857.416777] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5858.541753] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5859.668849] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5860.619418] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5861.681400] LustreError: 171977:0:(osp_precreate.c:655:osp_precreate_send()) lustre-OST0000-osc-MDT0000: can't precreate: rc = -5 [ 5861.688449] LustreError: 171977:0:(osp_precreate.c:1359:osp_precreate_thread()) lustre-OST0000-osc-MDT0000: cannot precreate objects: rc = -5 [ 5861.690200] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5862.702497] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5863.538481] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5864.456407] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5865.561093] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5866.597857] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5867.724157] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5868.860516] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5869.849897] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5871.217612] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5872.157439] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5873.175959] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5874.134996] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5875.052401] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5875.948563] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5876.925984] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5877.688649] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5878.659559] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5879.697408] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5880.586630] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5881.501496] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5882.665963] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5883.570877] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5884.502089] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5885.310904] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5886.241359] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5887.226753] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5888.181748] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5888.810485] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5889.661949] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5890.521081] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5891.403241] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5892.170760] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5892.966583] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5893.844536] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5894.939250] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5895.972329] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5897.087374] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5898.420362] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5899.747520] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5900.920099] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5901.921491] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5902.858447] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5903.792877] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5904.707732] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5905.797430] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5907.156853] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5908.221697] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5909.071496] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5909.892505] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5910.643586] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5911.581180] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5912.467477] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5913.334405] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5914.421860] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5915.429236] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5916.297820] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5917.387465] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5918.355582] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5919.380105] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5920.704120] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5921.732467] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5922.874941] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5924.312313] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5925.627199] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5926.952127] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5928.151602] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5929.530934] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5930.671148] LustreError: 171977:0:(osp_precreate.c:655:osp_precreate_send()) lustre-OST0000-osc-MDT0000: can't precreate: rc = -5 [ 5930.684170] LustreError: 171977:0:(osp_precreate.c:1359:osp_precreate_thread()) lustre-OST0000-osc-MDT0000: cannot precreate objects: rc = -5 [ 5930.700276] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5931.551432] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5932.600169] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5933.684507] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5934.635442] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5935.591700] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5936.256925] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5937.066200] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5938.039137] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5938.888393] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5939.622034] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5940.455966] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5941.305211] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5942.336293] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5943.236456] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5944.313907] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5945.320070] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5946.312229] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5947.351220] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5948.410309] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5949.499640] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5950.615473] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5951.514068] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5952.535271] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5953.546223] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5954.577708] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5955.520305] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5956.636909] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5957.505329] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5958.687492] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5959.596418] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5960.627939] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5961.530108] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5962.435168] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5963.254438] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5964.051820] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5964.984982] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5965.885362] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5966.857934] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5967.844755] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5968.741787] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5969.590043] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5970.395218] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5971.155380] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5971.995604] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5972.815843] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5973.763188] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5974.677616] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5975.719304] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5976.889529] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5977.984337] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5978.767918] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5979.635711] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5980.452578] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5981.401422] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5982.275544] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5983.371429] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5984.263415] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5985.228394] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5986.015891] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5986.868428] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5987.657873] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5988.535984] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5989.437308] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5990.323823] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5991.264946] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5992.025048] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5992.877048] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5993.748491] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5994.576207] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5995.641500] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5996.447117] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5997.444294] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5998.304258] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5999.137267] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5999.951817] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6000.769621] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6001.710238] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6002.619753] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6003.444167] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6004.342978] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6005.289456] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6006.104405] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6006.826811] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6007.620654] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6008.423159] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6009.176901] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6010.029679] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6010.775302] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6011.580623] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6012.314892] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6013.054961] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6013.905860] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6014.672258] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6015.582805] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6016.276992] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6017.082697] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6017.927801] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6018.916720] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6019.591505] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6020.415460] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6021.160842] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6021.963647] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6022.680497] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6023.518201] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6024.494129] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6025.342470] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6026.231635] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6027.074184] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6027.949973] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6028.890395] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6029.657846] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6030.365757] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6031.115083] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6031.918948] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6032.595779] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6033.360242] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6034.081453] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6034.891198] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6035.561250] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6036.369995] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6037.217078] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6038.050794] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6038.699078] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6039.452865] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6040.378844] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6041.423981] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6042.382482] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6043.735922] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6044.715580] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6045.823862] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6046.871861] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6048.035674] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6048.996923] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6050.148863] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6051.252662] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6052.099718] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6053.087943] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6054.121893] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6055.072979] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6056.247095] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6057.372662] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6058.589398] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6059.587684] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6060.399639] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6061.170851] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6061.939687] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6062.732633] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6063.553172] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6064.209286] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6064.946370] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6065.732759] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6066.464820] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6067.141755] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6067.977236] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6068.882256] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6069.625463] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6070.540285] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6071.485247] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6072.669535] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6073.454309] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6074.746701] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6075.896375] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6077.074895] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6078.403127] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6079.091262] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6080.039124] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6080.827033] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6082.203805] LustreError: 171977:0:(osp_precreate.c:655:osp_precreate_send()) lustre-OST0000-osc-MDT0000: can't precreate: rc = -5 [ 6082.228991] LustreError: 171977:0:(osp_precreate.c:1359:osp_precreate_thread()) lustre-OST0000-osc-MDT0000: cannot precreate objects: rc = -5 [ 6082.249258] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6083.237296] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6084.522066] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6085.991686] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6087.389571] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6088.951436] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6089.998415] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6091.250142] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6092.411878] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6093.400570] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6094.180328] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6095.191762] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6096.079054] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6097.027787] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6098.064930] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6098.887487] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6099.664473] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6100.533910] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6101.257504] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6102.064510] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6102.900579] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6103.734765] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6104.599433] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6105.432522] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6106.372641] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6107.169571] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6108.029761] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6108.774581] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6109.444715] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6109.987279] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6111.145370] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6111.939827] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6112.823703] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6113.632714] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6114.447559] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6115.271296] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6116.085216] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6116.987684] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6118.088361] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6118.938300] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6548.942508] LustreError: 173782:0:(ofd_dev.c:1780:ofd_create_hdl()) lustre-OST0000: unable to precreate: rc = -28 [ 6548.955336] LustreError: 171977:0:(osp_precreate.c:655:osp_precreate_send()) lustre-OST0000-osc-MDT0000: can't precreate: rc = -28 [ 6548.965261] LustreError: 171977:0:(osp_precreate.c:655:osp_precreate_send()) Skipped 2 previous similar messages [ 7304.675301] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 7304.679708] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 7304.688564] LustreError: Skipped 1 previous similar message [ 7304.706597] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 7304.712481] Lustre: Skipped 498 previous similar messages [ 7304.733031] Lustre: Skipped 1 previous similar message [ 7305.766835] Lustre: server umount lustre-OST0000 complete [ 7309.807641] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 7309.813116] Lustre: Skipped 1 previous similar message [ 7314.916879] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 7314.919564] Lustre: Skipped 1 previous similar message [ 7315.510942] Lustre: server umount lustre-MDT0000 complete [ 7318.449039] LustreError: 171938:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1788646015 with bad export cookie 10167816595773595155 [ 7318.451458] LustreError: MGC192.168.203.150@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 7318.463529] LustreError: 171938:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 7318.694514] Lustre: server umount lustre-MDT0001 complete [ 7329.056743] Lustre: DEBUG MARKER: oleg350-server.virtnet: executing unload_modules_local [ 7331.560993] Key type lgssc unregistered [ 7331.799478] LNet: 227259:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 7331.807739] LNetError: 227259:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 7331.831670] LNet: Removed LNI 192.168.203.150@tcp [ 7332.711548] Key type .llcrypt unregistered [ 7332.713198] Key type ._llcrypt unregistered [ 7362.648209] Key type ._llcrypt registered [ 7362.650517] Key type .llcrypt registered [ 7362.757756] Lustre: DEBUG MARKER: oleg350-server.virtnet: executing load_modules_local [ 7363.399238] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 7363.444089] alg: No test for adler32 (adler32-zlib) [ 7364.698125] Lustre: Lustre: Build Version: 2.17.58_3_g980e1e2 [ 7365.026465] LNet: Added LNI 192.168.203.150@tcp [8/256/0/180] [ 7366.855739] Key type lgssc registered [ 7367.878477] Lustre: Echo OBD driver; http://www.lustre.org/ [ 7375.280025] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 7375.301437] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 7376.618471] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 7379.921381] Lustre: DEBUG MARKER: oleg350-server.virtnet: executing set_default_debug -1 all [ 7386.098843] LustreError: 228673:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 7386.115788] LustreError: 228673:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 2 previous similar messages [ 7389.069919] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 7389.501427] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 7392.609599] Lustre: DEBUG MARKER: oleg350-server.virtnet: executing set_default_debug -1 all [ 7399.313562] Lustre: DEBUG MARKER: oleg350-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 7405.018929] Lustre: DEBUG MARKER: oleg350-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 7413.027376] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 7413.288238] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 7420.929611] Lustre: DEBUG MARKER: oleg350-server.virtnet: executing set_default_debug -1 all [ 7421.528806] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:49665 to 0x280000401:49696) [ 7429.101477] Lustre: DEBUG MARKER: oleg350-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 7434.347367] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 7435.860323] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 7435.879515] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 7435.888263] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7435.900034] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 7435.903026] LustreError: 228693:0:(osp_precreate.c:1268:osp_precreate_thread()) cfs_fail_timeout id 2107 sleeping for 20000ms [ 7439.469902] Lustre: server umount lustre-OST0000 complete [ 7455.199145] Lustre: 227984:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1788646136/real 1788646136] req@0000000096be3dc2 x1875531332109568/t0(0) o13->lustre-OST0000-osc-MDT0001@0@lo:7/4 lens 224/368 e 0 to 1 dl 1788646152 ref 1 fl Rpc:XQr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 [ 7455.229561] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 7455.975108] LustreError: 228693:0:(osp_precreate.c:1268:osp_precreate_thread()) cfs_fail_timeout id 2107 awake [ 7457.247148] Lustre: 227985:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1788646138/real 1788646138] req@0000000030240e90 x1875531332110592/t0(0) o400->lustre-OST0000-osc-MDT0001@0@lo:28/4 lens 224/224 e 0 to 1 dl 1788646154 ref 1 fl Rpc:XNQr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 [ 7457.311401] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 7462.431153] Lustre: 227987:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1788646143/real 1788646143] req@000000006d53aaef x1875531332111360/t0(0) o400->lustre-OST0000-osc-MDT0000@0@lo:28/4 lens 224/224 e 0 to 1 dl 1788646159 ref 1 fl Rpc:XNQr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 [ 7462.450138] Lustre: 227987:0:(client.c:2504:ptlrpc_expire_one_request()) Skipped 2 previous similar messages [ 7468.575485] Lustre: 227986:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1788646149/real 1788646149] req@0000000053eb6d15 x1875531332112896/t0(0) o400->lustre-OST0000-osc-MDT0001@0@lo:28/4 lens 224/224 e 0 to 1 dl 1788646165 ref 1 fl Rpc:XNQr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 [ 7468.614150] Lustre: 227986:0:(client.c:2504:ptlrpc_expire_one_request()) Skipped 1 previous similar message [ 7472.958879] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 7473.215744] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 7478.974711] Lustre: DEBUG MARKER: oleg350-server.virtnet: executing set_default_debug -1 all [ 7480.426877] LustreError: lustre-OST0000-osc-MDT0001: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7480.441414] Lustre: lustre-OST0000-osc-MDT0001: Connection restored to 0@lo (at 0@lo) [ 7482.476686] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7482.496100] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 7485.442221] Lustre: DEBUG MARKER: oleg350-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 7486.778741] Lustre: DEBUG MARKER: osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid in FULL state after 0 sec [ 7491.674473] Lustre: DEBUG MARKER: oleg350-client.virtnet: executing wait_import_state (FULL|IDLE) osc.lustre-OST0000-osc-ffff922258a45800.ost_server_uuid 50 [ 7493.047971] Lustre: DEBUG MARKER: osc.lustre-OST0000-osc-ffff922258a45800.ost_server_uuid in FULL state after 0 sec [ 7498.213928] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 7498.216552] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 7498.236267] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 7498.255956] Lustre: Skipped 1 previous similar message [ 7499.044719] Lustre: server umount lustre-OST0000 complete [ 7503.329233] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 7503.337888] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 7503.353190] Lustre: Skipped 2 previous similar messages [ 7503.365023] Lustre: Skipped 1 previous similar message [ 7508.450969] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 7508.455868] Lustre: Skipped 1 previous similar message [ 7508.692303] Lustre: server umount lustre-MDT0000 complete [ 7511.799820] LustreError: 233126:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1788646208 with bad export cookie 14786715545720687796 [ 7511.809549] LustreError: MGC192.168.203.150@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 7511.815269] LustreError: 233126:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 7512.072000] Lustre: server umount lustre-MDT0001 complete [ 7521.623539] Lustre: DEBUG MARKER: oleg350-server.virtnet: executing unload_modules_local [ 7525.068299] Key type lgssc unregistered [ 7525.365514] LNet: 233876:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 7525.374828] LNetError: 233876:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 7525.397895] LNet: Removed LNI 192.168.203.150@tcp [ 7526.445222] Key type .llcrypt unregistered [ 7526.451147] Key type ._llcrypt unregistered [ 7572.039490] Key type ._llcrypt registered [ 7572.042553] Key type .llcrypt registered [ 7572.233478] Lustre: DEBUG MARKER: oleg350-server.virtnet: executing load_modules_local [ 7573.478115] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 7573.540556] alg: No test for adler32 (adler32-zlib) [ 7574.716273] Lustre: Lustre: Build Version: 2.17.58_3_g980e1e2 [ 7574.978580] LNet: Added LNI 192.168.203.150@tcp [8/256/0/180] [ 7576.737677] Key type lgssc registered [ 7578.040329] Lustre: Echo OBD driver; http://www.lustre.org/ [ 7588.664519] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 7588.695929] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 7590.090168] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 7594.961492] Lustre: DEBUG MARKER: oleg350-server.virtnet: executing set_default_debug -1 all [ 7600.612638] LustreError: 235608:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 7600.635460] LustreError: 235608:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 2 previous similar messages [ 7604.484504] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 7604.862240] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 7609.453083] Lustre: DEBUG MARKER: oleg350-server.virtnet: executing set_default_debug -1 all [ 7613.363855] Lustre: Modifying parameter general.debug_raw_pointers=Y in log params [ 7620.756975] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 7621.290439] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 7631.189863] Lustre: DEBUG MARKER: oleg350-server.virtnet: executing set_default_debug -1 all [ 7635.720535] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:49698 to 0x280000401:49728) [ 7641.919391] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 7642.241967] Lustre: lustre-OST0001: new disk, initializing [ 7642.252132] Lustre: srv-lustre-OST0001: No data found on store. Initialize space. [ 7642.266772] Lustre: 237918:0:(osd_compat.c:1353:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 7642.367988] Lustre: lustre-OST0001: Imperative Recovery not enabled, recovery window 60-180 [ 7647.865737] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x00000002c0000400-0x0000000300000400]:1:ost [ 7647.888604] Lustre: cli-lustre-OST0001-super: Allocated super-sequence [0x00000002c0000400-0x0000000300000400]:1:ost] [ 7648.037347] Lustre: lustre-OST0001-osc-MDT0000: update sequence from 0x100010000 to 0x2c0000401 [ 7648.540730] Lustre: DEBUG MARKER: oleg350-server.virtnet: executing set_default_debug -1 all [ 7663.570748] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 7674.036111] Lustre: Setting parameter general.lod.*.mdt_hash=crush in log params [ 7691.749061] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 7691.763275] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 7691.780309] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 7694.304409] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 7694.311142] Lustre: lustre-MDT0000-lwp-OST0000: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 7694.320863] Lustre: Skipped 1 previous similar message [ 7697.268674] Lustre: server umount lustre-MDT0000 complete [ 7699.428071] LustreError: 235602:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 7699.451079] LustreError: 235602:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 3 previous similar messages [ 7700.928766] LustreError: 235588:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1788646397 with bad export cookie 15577205793682241412 [ 7700.933659] LustreError: MGC192.168.203.150@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 7700.946246] LustreError: 235588:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 2 previous similar messages [ 7701.431911] Lustre: server umount lustre-MDT0001 complete [ 7715.632396] Lustre: server umount lustre-OST0000 complete [ 7729.669227] Lustre: server umount lustre-OST0001 complete [ 7756.753546] Lustre: DEBUG MARKER: oleg350-server.virtnet: executing load_modules_local [ 7767.914492] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 7768.685220] LustreError: 241028:0:(ldlm_lib.c:1190:target_handle_connect()) mylustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 7768.821274] Lustre: mylustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 7774.187080] LustreError: 241029:0:(ldlm_lib.c:1190:target_handle_connect()) mylustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 7774.723064] Lustre: DEBUG MARKER: oleg350-server.virtnet: executing set_default_debug -1 all [ 7778.284838] LustreError: 241028:0:(ldlm_lib.c:1190:target_handle_connect()) mylustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 7785.155403] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 7785.670955] Lustre: mylustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 7791.110029] Lustre: DEBUG MARKER: oleg350-server.virtnet: executing set_default_debug -1 all [ 7794.814680] Lustre: 242171:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify general/debug_raw_pointers=Y (mode = 0) failed: rc = -17 [ 7802.180235] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 7802.695491] Lustre: mylustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 7810.687656] Lustre: DEBUG MARKER: oleg350-server.virtnet: executing set_default_debug -1 all [ 7810.930995] LustreError: 242524:0:(ldlm_lib.c:1190:target_handle_connect()) mylustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 7810.964235] LustreError: 242524:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 1 previous similar message [ 7814.019747] Lustre: mylustre-OST0000: new connection from mylustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:49731 to 0x280000401:49760) [ 7821.493621] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 7828.574549] Lustre: DEBUG MARKER: oleg350-server.virtnet: executing set_default_debug -1 all [ 7836.171678] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 7839.737837] Lustre: 244042:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify general/lod.*.mdt_hash=crush (mode = 0) failed: rc = -17 [ 7852.003952] LustreError: mylustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 7852.010886] Lustre: mylustre-MDT0000-osp-MDT0001: Connection to mylustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 7852.021500] Lustre: mylustre-MDT0000: Not available for connect from 0@lo (stopping) [ 7852.032772] Lustre: Skipped 3 previous similar messages [ 7855.929866] Lustre: server umount mylustre-MDT0000 complete [ 7857.635145] LustreError: 241028:0:(ldlm_lib.c:1190:target_handle_connect()) mylustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 7857.670785] LustreError: 241028:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 7 previous similar messages [ 7859.646708] LustreError: 243353:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1788646556 with bad export cookie 15577205793682244219 [ 7859.651408] LustreError: MGC192.168.203.150@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 7859.659181] LustreError: 243353:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 7859.919976] Lustre: server umount mylustre-MDT0001 complete [ 7873.831676] Lustre: server umount mylustre-OST0000 complete [ 7878.111357] Lustre: 234869:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1788646559/real 1788646559] req@ffff99a025c31f80 x1875531552422272/t0(0) o400->mylustre-MDT0001-lwp-OST0001@0@lo:12/10 lens 224/224 e 0 to 1 dl 1788646575 ref 1 fl Rpc:XNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 [ 7878.149408] Lustre: mylustre-MDT0001-lwp-OST0001: Connection to mylustre-MDT0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 7878.173038] Lustre: Skipped 3 previous similar messages [ 7878.713404] Lustre: server umount mylustre-OST0001 complete [ 7900.513257] Lustre: DEBUG MARKER: oleg350-server.virtnet: executing load_modules_local [ 7911.031511] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 7911.452875] LustreError: 246350:0:(ldlm_lib.c:1190:target_handle_connect()) tfs-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 7911.545776] Lustre: tfs-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 7911.552281] Lustre: Skipped 1 previous similar message [ 7915.938259] Lustre: DEBUG MARKER: oleg350-server.virtnet: executing set_default_debug -1 all [ 7923.997455] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 7929.266469] Lustre: DEBUG MARKER: oleg350-server.virtnet: executing set_default_debug -1 all [ 7932.263846] Lustre: 247491:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify general/debug_raw_pointers=Y (mode = 0) failed: rc = -17 [ 7938.650853] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 7945.474933] Lustre: DEBUG MARKER: oleg350-server.virtnet: executing set_default_debug -1 all [ 7954.716357] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 7956.093623] Lustre: tfs-OST0000: new connection from tfs-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:49731 to 0x280000401:49792) [ 7956.110066] Lustre: tfs-OST0001: new connection from tfs-MDT0000-mdtlov (cleaning up unused objects from 0x2c0000401:35 to 0x2c0000401:65) [ 7962.132544] Lustre: DEBUG MARKER: oleg350-server.virtnet: executing set_default_debug -1 all [ 7969.433625] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 7972.793076] Lustre: 249365:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify general/lod.*.mdt_hash=crush (mode = 0) failed: rc = -17 [ 7980.527595] LustreError: tfs-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 7980.536178] Lustre: tfs-MDT0000-osp-MDT0001: Connection to tfs-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 7980.569382] Lustre: tfs-MDT0000: Not available for connect from 0@lo (stopping) [ 7980.573117] Lustre: Skipped 3 previous similar messages [ 7984.676579] Lustre: server umount tfs-MDT0000 complete [ 7987.186405] LustreError: 246351:0:(ldlm_lib.c:1190:target_handle_connect()) tfs-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 7987.206297] LustreError: 246351:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 7 previous similar messages [ 7987.852995] LustreError: 247493:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1788646684 with bad export cookie 15577205793682245885 [ 7987.855529] LustreError: MGC192.168.203.150@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 7987.873575] LustreError: 247493:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 7988.116474] Lustre: server umount tfs-MDT0001 complete [ 8001.606514] Lustre: server umount tfs-OST0000 complete [ 8014.321378] Lustre: server umount tfs-OST0001 complete [ 8034.906980] Lustre: DEBUG MARKER: oleg350-server.virtnet: executing load_modules_local [ 8044.245167] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 8044.647088] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 8044.653481] Lustre: Skipped 3 previous similar messages [ 8048.415706] Lustre: DEBUG MARKER: oleg350-server.virtnet: executing set_default_debug -1 all [ 8057.493341] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 8061.908365] Lustre: DEBUG MARKER: oleg350-server.virtnet: executing set_default_debug -1 all [ 8064.659987] Lustre: 252758:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify general/debug_raw_pointers=Y (mode = 0) failed: rc = -17 [ 8070.545235] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 8076.104640] Lustre: DEBUG MARKER: oleg350-server.virtnet: executing set_default_debug -1 all [ 8083.690276] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 8089.093748] Lustre: DEBUG MARKER: oleg350-server.virtnet: executing set_default_debug -1 all [ 8090.097288] Lustre: lustre-OST0001: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x2c0000401:35 to 0x2c0000401:97) [ 8090.097847] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:49731 to 0x280000401:49824) [ 8095.184823] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 8098.380635] Lustre: 254630:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify general/lod.*.mdt_hash=crush (mode = 0) failed: rc = -17 [ 8105.218418] Lustre: DEBUG MARKER: == conf-sanity test 104a: Make sure user defined options are reflected in mount ========================================================== 18:20:00 (1788646800) [ 8120.802582] Lustre: lustre-MDT0000-lwp-OST0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 8120.805208] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 8120.816195] Lustre: Skipped 6 previous similar messages [ 8120.824781] Lustre: Skipped 7 previous similar messages [ 8126.909388] Lustre: server umount lustre-MDT0000 complete [ 8130.001822] LustreError: 251592:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1788646826 with bad export cookie 15577205793682247551 [ 8130.016823] LustreError: MGC192.168.203.150@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 8130.019596] LustreError: 251592:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 8131.040540] LustreError: 251610:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 8131.045117] Lustre: lustre-MDT0001: Not available for connect from 0@lo (stopping) [ 8131.058784] LustreError: 251610:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 5 previous similar messages [ 8131.070744] Lustre: Skipped 5 previous similar messages [ 8144.352414] Lustre: lustre-MDT0001 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 8144.611488] Lustre: server umount lustre-MDT0001 complete [ 8162.271277] Lustre: lustre-OST0000 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 8162.389744] Lustre: server umount lustre-OST0000 complete [ 8172.352336] Lustre: server umount lustre-OST0001 complete [ 8180.468880] Lustre: DEBUG MARKER: oleg350-server.virtnet: executing set_hostid [ 8192.082885] Lustre: DEBUG MARKER: oleg350-server.virtnet: executing load_modules_local [ 8222.429323] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 8222.591791] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 8222.614611] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 8222.682553] Lustre: lustre-MDT0000: new disk, initializing [ 8222.737976] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 188-180 [ 8222.745344] Lustre: Skipped 3 previous similar messages [ 8222.758184] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 8226.296120] Lustre: DEBUG MARKER: oleg350-server.virtnet: executing set_default_debug -1 all [ 8235.789316] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 8235.947773] Lustre: 257815:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 8236.017741] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [ 8236.021906] Lustre: Skipped 1 previous similar message [ 8236.104707] Lustre: lustre-MDT0001: new disk, initializing [ 8236.232319] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [ 8236.241806] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 8239.896578] Lustre: DEBUG MARKER: oleg350-server.virtnet: executing set_default_debug -1 all [ 8248.848423] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 8249.124045] Lustre: lustre-OST0000: new disk, initializing [ 8249.128737] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 8249.137870] Lustre: 259398:0:(osd_compat.c:1353:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 8250.750272] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [ 8250.757510] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 8250.875547] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 8254.687041] Lustre: DEBUG MARKER: oleg350-server.virtnet: executing set_default_debug -1 all [ 8263.203439] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 8263.320584] Lustre: lustre-OST0001: new disk, initializing [ 8263.323656] Lustre: srv-lustre-OST0001: No data found on store. Initialize space. [ 8263.332428] Lustre: 260267:0:(osd_compat.c:1353:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 8265.223868] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x00000002c0000400-0x0000000300000400]:1:ost [ 8265.237495] Lustre: cli-lustre-OST0001-super: Allocated super-sequence [0x00000002c0000400-0x0000000300000400]:1:ost] [ 8265.327728] Lustre: lustre-OST0001-osc-MDT0000: update sequence from 0x100010000 to 0x2c0000401 [ 8268.325409] Lustre: DEBUG MARKER: oleg350-server.virtnet: executing set_default_debug -1 all [ 8279.546673] Lustre: DEBUG MARKER: == conf-sanity test 104b: Mount uses last flock argument ========================================================== 18:22:55 (1788646975) [ 8291.149194] Lustre: DEBUG MARKER: == conf-sanity test 105: check file creation for ro and rw bind mnt pt ========================================================== 18:23:06 (1788646986) [ 8306.152128] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 8306.161704] Lustre: Skipped 2 previous similar messages [ 8306.166236] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 8306.174606] Lustre: Skipped 7 previous similar messages [ 8311.041209] Lustre: server umount lustre-OST0000 complete [ 8322.725188] Lustre: server umount lustre-MDT0000 complete [ 8326.391130] LustreError: 257807:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1788647023 with bad export cookie 15577205793682249098 [ 8326.398431] LustreError: MGC192.168.203.150@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 8326.410844] LustreError: 257807:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 8332.782885] Lustre: server umount lustre-MDT0001 complete [ 8344.699633] Lustre: DEBUG MARKER: oleg350-server.virtnet: executing unload_modules_local [ 8422.622105] Lustre: server umount lustre-OST0001 complete [ 8429.203853] Lustre: DEBUG MARKER: oleg350-server.virtnet: executing set_hostid [ 8443.518146] Lustre: DEBUG MARKER: oleg350-server.virtnet: executing load_modules_local [ 8475.218517] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 8475.518394] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 8475.556960] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 8475.657976] Lustre: lustre-MDT0000: new disk, initializing [ 8475.795979] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 8479.784945] Lustre: DEBUG MARKER: oleg350-server.virtnet: executing set_default_debug -1 all [ 8490.151267] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 8490.275205] Lustre: 265122:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 8490.511716] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 8490.517247] Lustre: Skipped 4 previous similar messages [ 8490.554187] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 8494.242757] Lustre: DEBUG MARKER: oleg350-server.virtnet: executing set_default_debug -1 all [ 8503.709779] Lustre: DEBUG MARKER: oleg350-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 8508.359767] Lustre: DEBUG MARKER: oleg350-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 8513.776907] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 8514.112636] Lustre: lustre-OST0000: new disk, initializing [ 8514.115420] Lustre: Skipped 1 previous similar message [ 8514.117688] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 8514.120989] Lustre: Skipped 2 previous similar messages [ 8514.124426] Lustre: 267092:0:(osd_compat.c:1353:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 8515.761245] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [ 8515.767875] Lustre: Skipped 1 previous similar message [ 8515.770819] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 8515.875561] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 8520.227770] Lustre: DEBUG MARKER: oleg350-server.virtnet: executing set_default_debug -1 all [ 8529.989383] Lustre: DEBUG MARKER: oleg350-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 8541.667703] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 8541.673656] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 8541.676977] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 8541.676986] Lustre: Skipped 5 previous similar messages [ 8541.679501] LustreError: Skipped 1 previous similar message [ 8541.698695] Lustre: Skipped 6 previous similar messages [ 8543.604378] Lustre: server umount lustre-OST0000 complete [ 8546.784794] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 8554.926723] Lustre: server umount lustre-MDT0000 complete [ 8557.030792] LustreError: 265136:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 8557.057510] LustreError: 265136:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 16 previous similar messages [ 8559.507925] LustreError: 267091:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1788647256 with bad export cookie 15577205793682250575 [ 8559.523306] LustreError: MGC192.168.203.150@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 8559.539744] LustreError: 267091:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 8560.017046] Lustre: server umount lustre-MDT0001 complete [ 8572.849745] Lustre: DEBUG MARKER: oleg350-server.virtnet: executing unload_modules_local [ 8575.771244] Key type lgssc unregistered [ 8576.066436] LNet: 268850:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 8576.071169] LNetError: 268850:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 8576.080821] LNet: Removed LNI 192.168.203.150@tcp [ 8576.892153] Key type .llcrypt unregistered [ 8576.895566] Key type ._llcrypt unregistered [ 8606.661769] Key type ._llcrypt registered [ 8606.668446] Key type .llcrypt registered [ 8606.791581] Lustre: DEBUG MARKER: oleg350-server.virtnet: executing load_modules_local [ 8607.873103] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 8607.885200] alg: No test for adler32 (adler32-zlib) [ 8608.968603] Lustre: Lustre: Build Version: 2.17.58_3_g980e1e2 [ 8609.257471] LNet: Added LNI 192.168.203.150@tcp [8/256/0/180] [ 8610.943341] Key type lgssc registered [ 8612.243677] Lustre: Echo OBD driver; http://www.lustre.org/ [ 8619.982672] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 8620.008051] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 8621.483566] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 8625.748309] Lustre: DEBUG MARKER: oleg350-server.virtnet: executing set_default_debug -1 all [ 8631.786344] LustreError: 270307:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 8631.814251] LustreError: 270307:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 2 previous similar messages [ 8636.475973] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 8636.939549] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 8641.887151] Lustre: DEBUG MARKER: oleg350-server.virtnet: executing set_default_debug -1 all [ 8649.727582] Lustre: DEBUG MARKER: oleg350-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 8655.686700] Lustre: DEBUG MARKER: oleg350-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 8661.794217] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 8662.264164] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 8666.347440] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:3 to 0x280000401:33) [ 8667.751940] Lustre: DEBUG MARKER: oleg350-server.virtnet: executing set_default_debug -1 all [ 8674.666857] Lustre: DEBUG MARKER: oleg350-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 8675.612157] Lustre: Setting parameter lustre-OST0000.ost.unknown_param=50 in log lustre-OST0000 [ 8681.390422] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 8681.391379] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 8683.570623] Lustre: server umount lustre-OST0000 complete [ 8688.100623] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 8688.108339] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 8688.121686] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 8688.131202] Lustre: Skipped 1 previous similar message [ 8691.681951] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 8691.683095] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 8693.582915] Lustre: server umount lustre-MDT0000 complete [ 8696.802400] LustreError: 270303:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 8696.809944] LustreError: 270303:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 1 previous similar message [ 8696.905163] LustreError: 270288:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1788647393 with bad export cookie 16042116730050991785 [ 8696.905991] LustreError: MGC192.168.203.150@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 8696.914971] LustreError: 270288:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 8697.208533] Lustre: server umount lustre-MDT0001 complete [ 8709.160294] Lustre: DEBUG MARKER: oleg350-server.virtnet: executing unload_modules_local [ 8712.435066] Key type lgssc unregistered [ 8712.684609] LNet: 273894:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 8712.694439] LNetError: 273894:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 8712.731832] LNet: Removed LNI 192.168.203.150@tcp [ 8714.040480] Key type .llcrypt unregistered [ 8714.043413] Key type ._llcrypt unregistered [ 8731.961561] Key type ._llcrypt registered [ 8731.963825] Key type .llcrypt registered [ 8732.099528] Lustre: DEBUG MARKER: oleg350-server.virtnet: executing load_modules_local [ 8733.217125] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 8733.241738] alg: No test for adler32 (adler32-zlib) [ 8734.303752] Lustre: Lustre: Build Version: 2.17.58_3_g980e1e2 [ 8734.501726] LNet: Added LNI 192.168.203.150@tcp [8/256/0/180] [ 8736.183385] Key type lgssc registered [ 8737.194220] Lustre: Echo OBD driver; http://www.lustre.org/ [ 8745.314106] Lustre: lustre-OST0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 8745.345862] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 8760.991394] LustreError: Failed to get MGS log lustre-sptlrpc and no local copy. [ 8766.111452] LustreError: 275088:0:(mgc_request.c:233:do_config_log_add()) MGC192.168.203.150@tcp: failed processing log, type 4: rc = -110 [ 8792.799326] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 8799.083842] Lustre: DEBUG MARKER: oleg350-server.virtnet: executing set_default_debug -1 all [ 8806.699321] Lustre: DEBUG MARKER: oleg350-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 8809.023345] Lustre: server umount lustre-OST0000 complete [ 8826.449801] Lustre: DEBUG MARKER: oleg350-server.virtnet: executing unload_modules_local [ 8829.186621] Key type lgssc unregistered [ 8829.638221] LNet: 276719:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 8829.665208] LNetError: 276719:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 8829.697933] LNet: Removed LNI 192.168.203.150@tcp [ 8830.515612] Key type .llcrypt unregistered [ 8830.520273] Key type ._llcrypt unregistered [ 8874.566538] Key type ._llcrypt registered [ 8874.568942] Key type .llcrypt registered [ 8874.700938] Lustre: DEBUG MARKER: oleg350-server.virtnet: executing load_modules_local [ 8875.433880] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 8875.474651] alg: No test for adler32 (adler32-zlib) [ 8876.649214] Lustre: Lustre: Build Version: 2.17.58_3_g980e1e2 [ 8876.952054] LNet: Added LNI 192.168.203.150@tcp [8/256/0/180] [ 8878.663239] Key type lgssc registered [ 8880.359898] Lustre: Echo OBD driver; http://www.lustre.org/ [ 8893.859273] loop0: detected capacity change from 0 to 204800000 [ 8893.977112] blk_update_request: operation not supported error, dev loop0, sector 399872 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [ 8893.994895] blk_update_request: operation not supported error, dev loop0, sector 208 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [ 8894.332228] blk_update_request: operation not supported error, dev loop0, sector 160232 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [ 8895.387404] loop0: detected capacity change from 0 to 204800000 [ 8895.584483] blk_update_request: operation not supported error, dev loop0, sector 399872 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [ 8895.601111] blk_update_request: operation not supported error, dev loop0, sector 208 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [ 8895.850300] blk_update_request: operation not supported error, dev loop0, sector 160232 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [ 8897.073601] loop0: detected capacity change from 0 to 204800000 [ 8897.250481] blk_update_request: operation not supported error, dev loop0, sector 399872 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [ 8897.263925] blk_update_request: operation not supported error, dev loop0, sector 8224 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [ 8897.425773] blk_update_request: operation not supported error, dev loop0, sector 58264 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [ 8898.499921] loop0: detected capacity change from 0 to 204800000 [ 8898.615779] blk_update_request: operation not supported error, dev loop0, sector 399872 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [ 8899.606121] loop0: detected capacity change from 0 to 204800000 [ 8899.664539] LDISKFS-fs (loop0): mounted filesystem with ordered data mode. Opts: (null) [ 8903.464881] loop0: detected capacity change from 0 to 204800000 [ 8903.488170] LDISKFS-fs (loop0): mounted filesystem with ordered data mode. Opts: (null) [ 8907.125534] loop0: detected capacity change from 0 to 204800000 [ 8907.159822] LDISKFS-fs (loop0): mounted filesystem with ordered data mode. Opts: (null) [ 8911.585815] loop0: detected capacity change from 0 to 204800000 [ 8911.687232] LDISKFS-fs (loop0): mounted filesystem with ordered data mode. Opts: (null) [ 8917.241488] loop0: detected capacity change from 0 to 204800000 [ 8917.390767] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 8917.455763] LDISKFS-fs (loop0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 8917.578943] Lustre: 280129:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0000: restore index 'fld' with [0x200000001:0x3:0x0]: rc = 0 [ 8917.595656] Lustre: 280129:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0000: restore index 'nodemap' with [0x200000003:0x3:0x0]: rc = 0 [ 8917.603439] Lustre: 280129:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0000: restore index '0x10000' with [0x200000003:0xa:0x0]: rc = 0 [ 8917.614210] Lustre: 280129:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0000: restore index '0x10000-MDT0000' with [0x200000005:0x1:0x0]: rc = 0 [ 8917.627613] Lustre: 280129:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0000: restore index '0x1010000-MDT0000' with [0x200000005:0x2:0x0]: rc = 0 [ 8917.636707] Lustre: 280129:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0000: restore index '0x2010000' with [0x200000003:0xc:0x0]: rc = 0 [ 8917.651815] Lustre: 280129:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0000: restore index '0x2010000-MDT0000' with [0x200000005:0x3:0x0]: rc = 0 [ 8917.666824] Lustre: 280129:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0000: restore index '0x1010000' with [0x200000003:0xb:0x0]: rc = 0 [ 8917.679485] Lustre: 280129:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0000: restore index '0x10000' with [0x200000006:0x10000:0x0]: rc = 0 [ 8917.689155] Lustre: 280129:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0000: restore index '0x2010000' with [0x200000006:0x2010000:0x0]: rc = 0 [ 8917.711372] Lustre: 280129:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0000: restore index '0x1010000' with [0x200000006:0x1010000:0x0]: rc = 0 [ 8917.733855] Lustre: 280129:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0000: restore index '0x20000' with [0x200000006:0x20000:0x0]: rc = 0 [ 8917.746601] Lustre: 280129:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0000: restore index '0x1020000' with [0x200000006:0x1020000:0x0]: rc = 0 [ 8917.758661] Lustre: 280129:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0000: restore index '0x2020000' with [0x200000006:0x2020000:0x0]: rc = 0 [ 8925.552987] Lustre: server umount MGS complete [ 8926.411219] loop0: detected capacity change from 0 to 204800000 [ 8926.485790] LDISKFS-fs (loop0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 8927.053635] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 8927.108461] Lustre: lustre-MDT0000: Aborting client recovery [ 8928.046154] loop1: detected capacity change from 0 to 204800000 [ 8928.220362] LDISKFS-fs (loop1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 8928.342293] Lustre: 280563:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0001: restore index 'fld' with [0x200000001:0x3:0x0]: rc = 0 [ 8928.363325] Lustre: 280563:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0001: restore index '0x1010000-MDT0001' with [0x200000005:0x2:0x0]: rc = 0 [ 8928.376482] Lustre: 280563:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0001: restore index '0x1010000' with [0x200000003:0x5:0x0]: rc = 0 [ 8928.395636] Lustre: 280563:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0001: restore index '0x2010000' with [0x200000003:0x6:0x0]: rc = 0 [ 8928.406513] Lustre: 280563:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0001: restore index '0x10000' with [0x200000003:0x4:0x0]: rc = 0 [ 8928.421693] Lustre: 280563:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0001: restore index '0x2010000-MDT0001' with [0x200000005:0x3:0x0]: rc = 0 [ 8928.440407] Lustre: 280563:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0001: restore index '0x10000-MDT0001' with [0x200000005:0x1:0x0]: rc = 0 [ 8928.476722] Lustre: Found index 1 for lustre-MDT0001, updating log [ 8928.730893] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 8928.807475] Lustre: lustre-MDT0001: Aborting client recovery [ 8929.900726] loop2: detected capacity change from 0 to 204800000 [ 8930.068929] LDISKFS-fs (loop2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 8930.216979] Lustre: 280640:0:(osd_scrub.c:1854:osd_index_restore()) lustre-OST0000: restore index 'fld' with [0x200000001:0x3:0x0]: rc = 0 [ 8930.238099] Lustre: 280640:0:(osd_scrub.c:1854:osd_index_restore()) lustre-OST0000: restore index '0x20000-OST0000' with [0x200000005:0x1:0x0]: rc = 0 [ 8930.266217] Lustre: 280640:0:(osd_scrub.c:1854:osd_index_restore()) lustre-OST0000: restore index '0x2020000' with [0x200000003:0x7:0x0]: rc = 0 [ 8930.286549] Lustre: 280640:0:(osd_scrub.c:1854:osd_index_restore()) lustre-OST0000: restore index '0x2020000-OST0000' with [0x200000005:0x3:0x0]: rc = 0 [ 8930.299167] Lustre: 280640:0:(osd_scrub.c:1854:osd_index_restore()) lustre-OST0000: restore index '0x1020000-OST0000' with [0x200000005:0x2:0x0]: rc = 0 [ 8930.308471] Lustre: 280640:0:(osd_scrub.c:1854:osd_index_restore()) lustre-OST0000: restore index '0x1020000' with [0x200000003:0x6:0x0]: rc = 0 [ 8930.319848] Lustre: 280640:0:(osd_scrub.c:1854:osd_index_restore()) lustre-OST0000: restore index '0x20000' with [0x200000003:0x5:0x0]: rc = 0 [ 8930.446300] Lustre: Found index 0 for lustre-OST0000, updating log [ 8930.510612] Lustre: 280659:0:(osd_compat.c:1353:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 8930.591318] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 8930.606155] Lustre: lustre-OST0000: Aborting recovery [ 8931.334392] loop3: detected capacity change from 0 to 204800000 [ 8931.501734] LDISKFS-fs (loop3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 8931.575293] Lustre: 280725:0:(osd_scrub.c:1854:osd_index_restore()) lustre-OST0001: restore index 'fld' with [0x200000001:0x3:0x0]: rc = 0 [ 8931.588592] Lustre: 280725:0:(osd_scrub.c:1854:osd_index_restore()) lustre-OST0001: restore index '0x20000' with [0x200000003:0x5:0x0]: rc = 0 [ 8931.597624] Lustre: 280725:0:(osd_scrub.c:1854:osd_index_restore()) lustre-OST0001: restore index '0x2020000-OST0001' with [0x200000005:0x3:0x0]: rc = 0 [ 8931.609200] Lustre: 280725:0:(osd_scrub.c:1854:osd_index_restore()) lustre-OST0001: restore index '0x1020000-OST0001' with [0x200000005:0x2:0x0]: rc = 0 [ 8931.624518] Lustre: 280725:0:(osd_scrub.c:1854:osd_index_restore()) lustre-OST0001: restore index '0x2020000' with [0x200000003:0x7:0x0]: rc = 0 [ 8931.634838] Lustre: 280725:0:(osd_scrub.c:1854:osd_index_restore()) lustre-OST0001: restore index '0x20000-OST0001' with [0x200000005:0x1:0x0]: rc = 0 [ 8931.645128] Lustre: 280725:0:(osd_scrub.c:1854:osd_index_restore()) lustre-OST0001: restore index '0x1020000' with [0x200000003:0x6:0x0]: rc = 0 [ 8931.683503] Lustre: Found index 1 for lustre-OST0001, updating log [ 8931.718148] Lustre: 280730:0:(osd_compat.c:1353:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 8931.808436] Lustre: lustre-OST0001: Aborting recovery [ 8936.940601] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x0:6 to 0x0:33) [ 8936.961351] Lustre: lustre-OST0001: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x0:6 to 0x0:33) [ 8936.973218] LustreError: 280506:0:(lod_dev.c:511:lod_sub_recovery_thread()) lustre-MDT0001-osp-MDT0000: get update log duration 9, retries 0, failed: rc = -108 [ 8936.991100] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000bd0 [ 8937.031312] Lustre: lustre-OST0001-osc-MDT0000: update sequence from 0x100010000 to 0x2c0000bd0 [ 8939.003677] Lustre: lustre-OST0000: new connection from lustre-MDT0001-mdtlov (cleaning up unused objects from 0x280000400:5 to 0x280000400:33) [ 8939.013654] Lustre: lustre-OST0001: new connection from lustre-MDT0001-mdtlov (cleaning up unused objects from 0x2c0000400:5 to 0x2c0000400:33) [ 8939.817530] LustreError: 280472:0:(mdt_identity.c:115:mdt_identity_do_upcall()) lustre-MDT0000: error invoking upcall /root/Work/Lustre/R5/lustre-release/lustre/tests/../utils/l_getidentity lustre-MDT0000 0: rc -2; check /proc/fs/lustre/mdt/lustre-MDT0000/identity_upcall, time 5729us: rc = -2 [ 8940.322819] LustreError: 280469:0:(mdt_identity.c:115:mdt_identity_do_upcall()) lustre-MDT0000: error invoking upcall /root/Work/Lustre/R5/lustre-release/lustre/tests/../utils/l_getidentity lustre-MDT0000 0: rc -2; check /proc/fs/lustre/mdt/lustre-MDT0000/identity_upcall, time 561us: rc = -2 [ 8940.354605] LustreError: 280469:0:(mdt_identity.c:115:mdt_identity_do_upcall()) Skipped 19 previous similar messages [ 8954.336285] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 8954.340092] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 8954.342154] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 8954.384870] Lustre: Skipped 2 previous similar messages [ 8959.458867] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 8959.485671] Lustre: Skipped 6 previous similar messages [ 8964.577099] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 8964.585862] Lustre: Skipped 3 previous similar messages [ 8967.135208] Lustre: lustre-MDT0000 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 8967.387905] Lustre: server umount lustre-MDT0000 complete [ 8968.605107] LustreError: 280455:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1788647665 with bad export cookie 7632480841260048318 [ 8968.620750] LustreError: MGC192.168.203.150@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 8968.626714] LustreError: 280455:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 3 previous similar messages [ 8969.697358] LustreError: 280471:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 8969.704327] Lustre: lustre-MDT0001-lwp-OST0001: Connection to lustre-MDT0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 8969.705321] Lustre: lustre-MDT0001: Not available for connect from 0@lo (stopping) [ 8969.723820] LustreError: 280471:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 3 previous similar messages [ 8969.740549] Lustre: Skipped 2 previous similar messages [ 8973.729254] LustreError: 281070:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 8973.729556] Lustre: lustre-MDT0001: Not available for connect from 0@lo (stopping) [ 8973.756065] LustreError: 281070:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 1 previous similar message [ 8973.764732] Lustre: Skipped 2 previous similar messages [ 8975.126871] Lustre: server umount lustre-MDT0001 complete [ 8982.313052] Lustre: server umount lustre-OST0000 complete [ 8990.037384] Lustre: server umount lustre-OST0001 complete [ 8999.340158] Lustre: DEBUG MARKER: == conf-sanity test 109a: test lctl clear_conf fsname ==== 18:34:54 (1788647694) [ 9027.296028] Lustre: DEBUG MARKER: oleg350-server.virtnet: executing set_hostid [ 9037.000273] Lustre: DEBUG MARKER: oleg350-server.virtnet: executing load_modules_local [ 9078.529563] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 9079.050196] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 9079.158482] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 9079.387856] Lustre: lustre-MDT0000: new disk, initializing [ 9079.531488] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 9079.539655] Lustre: Skipped 1 previous similar message [ 9079.581536] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 9084.231773] Lustre: DEBUG MARKER: oleg350-server.virtnet: executing set_default_debug -1 all [ 9095.336180] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 9095.470482] Lustre: 284802:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 9095.518214] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [ 9095.530683] Lustre: Skipped 1 previous similar message [ 9095.625548] Lustre: lustre-MDT0001: new disk, initializing [ 9095.730673] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 9095.773312] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [ 9095.789987] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 9100.156634] Lustre: DEBUG MARKER: oleg350-server.virtnet: executing set_default_debug -1 all [ 9109.898215] Lustre: DEBUG MARKER: oleg350-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 9115.371916] Lustre: DEBUG MARKER: oleg350-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 9122.256034] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 9122.586692] Lustre: lustre-OST0000: new disk, initializing [ 9122.595837] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 9122.609146] Lustre: 286769:0:(osd_compat.c:1353:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 9122.735311] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 9128.950806] Lustre: DEBUG MARKER: oleg350-server.virtnet: executing set_default_debug -1 all [ 9129.548231] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [ 9129.565870] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 9129.703328] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 9138.571600] Lustre: DEBUG MARKER: oleg350-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 9143.870139] Lustre: Setting parameter lustre-MDT0000.mdd.atime_diff=62 in log lustre-MDT0000 [ 9155.381758] Lustre: Modifying parameter lustre-MDT0000.mdd.atime_diff=63 in log lustre-MDT0000 [ 9159.526980] Lustre: Setting parameter lustre-client.llite.max_read_ahead_mb=32 in log lustre-client [ 9165.259949] Lustre: Modifying parameter lustre-client.llite.max_read_ahead_mb=64 in log lustre-client [ 9206.755826] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 9206.756133] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 9206.762662] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 9206.782902] Lustre: Skipped 1 previous similar message [ 9206.819578] Lustre: Skipped 1 previous similar message [ 9211.200383] Lustre: server umount lustre-OST0000 complete [ 9217.006269] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 9217.026380] Lustre: Skipped 1 previous similar message [ 9221.696713] Lustre: server umount lustre-MDT0000 complete [ 9222.112693] LustreError: 284814:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 9222.160467] LustreError: 284814:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 1 previous similar message [ 9225.742967] LustreError: 287624:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1788647922 with bad export cookie 7632480841260051041 [ 9225.744316] LustreError: MGC192.168.203.150@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 9225.756238] LustreError: 287624:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 9226.088981] Lustre: server umount lustre-MDT0001 complete [ 9233.576102] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 9238.277959] Lustre: DEBUG MARKER: oleg350-server.virtnet: executing set_default_debug -1 all [ 9244.191619] Lustre: server umount MGS complete [ 9258.545827] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 9259.239520] LustreError: 290587:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 9259.283294] LustreError: 290600:0:(qmt_pool.c:1639:qmt_pool_add_rem()) lustre-QMT0000: can't remove lustre-OST0000_UUID pool 'pool1': rc = -22 [ 9259.374752] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 9263.813098] Lustre: DEBUG MARKER: oleg350-server.virtnet: executing set_default_debug -1 all [ 9264.612410] LustreError: 290588:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 9270.953600] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 9275.032288] Lustre: DEBUG MARKER: oleg350-server.virtnet: executing set_default_debug -1 all [ 9283.060980] Lustre: DEBUG MARKER: oleg350-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 9289.971527] Lustre: DEBUG MARKER: oleg350-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 9298.488549] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 9298.738764] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 9298.746172] Lustre: Skipped 1 previous similar message [ 9304.147756] Lustre: DEBUG MARKER: oleg350-server.virtnet: executing set_default_debug -1 all [ 9311.162551] Lustre: DEBUG MARKER: oleg350-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 9340.896236] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 9340.900895] LustreError: Skipped 1 previous similar message [ 9340.908219] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 9340.923185] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 9340.928912] Lustre: Skipped 2 previous similar messages [ 9346.639596] Lustre: server umount lustre-OST0000 complete [ 9352.169633] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 9352.195835] Lustre: Skipped 2 previous similar messages [ 9357.689377] Lustre: server umount lustre-MDT0000 complete [ 9361.380163] LustreError: 290582:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 9361.406421] LustreError: 290582:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 2 previous similar messages [ 9362.113547] LustreError: 290569:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1788648059 with bad export cookie 7632480841260052868 [ 9362.118495] LustreError: MGC192.168.203.150@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 9362.133242] LustreError: 290569:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 9362.596710] Lustre: server umount lustre-MDT0001 complete [ 9375.188318] Lustre: DEBUG MARKER: oleg350-server.virtnet: executing unload_modules_local [ 9377.915602] Key type lgssc unregistered [ 9378.201751] LNet: 294530:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 9378.213154] LNetError: 294530:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 9378.249694] LNet: Removed LNI 192.168.203.150@tcp [ 9379.223174] Key type .llcrypt unregistered [ 9379.225362] Key type ._llcrypt unregistered [ 9415.560787] Key type ._llcrypt registered [ 9415.564544] Key type .llcrypt registered [ 9415.690514] Lustre: DEBUG MARKER: oleg350-server.virtnet: executing set_hostid [ 9429.871461] Lustre: DEBUG MARKER: oleg350-server.virtnet: executing load_modules_local [ 9430.468669] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 9430.522465] alg: No test for adler32 (adler32-zlib) [ 9431.540493] Lustre: Lustre: Build Version: 2.17.58_3_g980e1e2 [ 9431.760161] LNet: Added LNI 192.168.203.150@tcp [8/256/0/180] [ 9433.407238] Key type lgssc registered [ 9434.535240] Lustre: Echo OBD driver; http://www.lustre.org/ [ 9473.273751] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 9473.318158] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 9474.632276] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 9474.653120] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 9474.746485] Lustre: lustre-MDT0000: new disk, initializing [ 9474.818915] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 9474.840047] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 9479.352516] Lustre: DEBUG MARKER: oleg350-server.virtnet: executing set_default_debug -1 all [ 9489.510781] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 9489.595506] Lustre: 298486:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 9489.655930] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [ 9489.659332] Lustre: Skipped 1 previous similar message [ 9489.711979] Lustre: lustre-MDT0001: new disk, initializing [ 9489.765964] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 9489.804980] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [ 9489.811203] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 9493.965803] Lustre: DEBUG MARKER: oleg350-server.virtnet: executing set_default_debug -1 all [ 9502.595871] Lustre: DEBUG MARKER: oleg350-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 9507.315136] Lustre: DEBUG MARKER: oleg350-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 9512.468098] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 9512.737634] Lustre: lustre-OST0000: new disk, initializing [ 9512.742061] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 9512.750331] Lustre: 300456:0:(osd_compat.c:1353:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 9512.875738] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 9519.691253] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [ 9519.710164] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 9519.898164] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 9520.069221] Lustre: DEBUG MARKER: oleg350-server.virtnet: executing set_default_debug -1 all [ 9530.062923] Lustre: DEBUG MARKER: oleg350-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 9538.318648] Lustre: Setting parameter lustre-MDT0000.mdd.atime_diff=62 in log lustre-MDT0000 [ 9550.277454] Lustre: Modifying parameter lustre-MDT0000.mdd.atime_diff=63 in log lustre-MDT0000 [ 9558.818525] Lustre: Setting parameter lustre-client.llite.max_read_ahead_mb=32 in log lustre-client [ 9565.587632] Lustre: Modifying parameter lustre-client.llite.max_read_ahead_mb=64 in log lustre-client [ 9607.139411] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 9607.148766] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 9607.160242] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 9607.170078] Lustre: Skipped 1 previous similar message [ 9612.537303] Lustre: server umount lustre-OST0000 complete [ 9617.385687] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 9617.402203] Lustre: Skipped 1 previous similar message [ 9617.406082] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 9617.423407] Lustre: Skipped 1 previous similar message [ 9622.498822] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 9622.504694] Lustre: Skipped 1 previous similar message [ 9622.752702] Lustre: server umount lustre-MDT0000 complete [ 9626.560551] LustreError: 298480:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1788648323 with bad export cookie 6992724265808025628 [ 9626.567557] LustreError: MGC192.168.203.150@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 9626.576656] LustreError: 298480:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 9626.890260] Lustre: server umount lustre-MDT0001 complete [ 9634.815278] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 9638.835874] Lustre: DEBUG MARKER: oleg350-server.virtnet: executing set_default_debug -1 all [ 9643.423850] Lustre: server umount MGS complete [ 9654.463204] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 9654.834739] LustreError: 304374:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 9654.854608] LustreError: 304374:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 3 previous similar messages [ 9654.872335] LustreError: 304387:0:(qmt_pool.c:1639:qmt_pool_add_rem()) lustre-QMT0000: can't remove lustre-OST0000_UUID pool 'pool1': rc = -22 [ 9654.990620] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 9658.972826] Lustre: DEBUG MARKER: oleg350-server.virtnet: executing set_default_debug -1 all [ 9660.386290] LustreError: 304375:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 9664.486531] LustreError: 304374:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 9667.588644] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 9667.882506] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 9671.168580] Lustre: DEBUG MARKER: oleg350-server.virtnet: executing set_default_debug -1 all [ 9677.732734] Lustre: DEBUG MARKER: oleg350-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 9683.280772] Lustre: DEBUG MARKER: oleg350-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 9690.374959] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 9690.649215] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 9697.006724] Lustre: DEBUG MARKER: oleg350-server.virtnet: executing set_default_debug -1 all [ 9704.145273] Lustre: DEBUG MARKER: oleg350-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 9735.651702] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 9735.657001] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 9735.657343] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 9735.662443] Lustre: Skipped 1 previous similar message [ 9740.071837] Lustre: server umount lustre-OST0000 complete [ 9744.865991] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 9744.871170] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 9744.883618] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 9744.887433] Lustre: Skipped 1 previous similar message [ 9749.992973] Lustre: server umount lustre-MDT0000 complete [ 9751.009220] LustreError: 304375:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 9751.032098] LustreError: 304375:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 1 previous similar message [ 9752.740335] LustreError: 307105:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1788648449 with bad export cookie 6992724265808027567 [ 9752.743254] LustreError: MGC192.168.203.150@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 9752.745610] LustreError: 307105:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 9752.999831] Lustre: server umount lustre-MDT0001 complete [ 9762.286100] Lustre: DEBUG MARKER: oleg350-server.virtnet: executing unload_modules_local [ 9765.307702] Key type lgssc unregistered [ 9765.624928] LNet: 308323:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 9765.653377] LNetError: 308323:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 9765.676256] LNet: Removed LNI 192.168.203.150@tcp [ 9766.624830] Key type .llcrypt unregistered [ 9766.631521] Key type ._llcrypt unregistered [ 9786.828137] Key type ._llcrypt registered [ 9786.831830] Key type .llcrypt registered [ 9786.922893] Lustre: DEBUG MARKER: oleg350-server.virtnet: executing set_hostid [ 9801.908833] Lustre: DEBUG MARKER: oleg350-server.virtnet: executing load_modules_local [ 9803.288233] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 9803.377530] alg: No test for adler32 (adler32-zlib) [ 9804.662680] Lustre: Lustre: Build Version: 2.17.58_3_g980e1e2 [ 9804.915064] LNet: Added LNI 192.168.203.150@tcp [8/256/0/180] [ 9806.567171] Key type lgssc registered [ 9807.362169] Lustre: Echo OBD driver; http://www.lustre.org/ [ 9827.121283] Lustre: DEBUG MARKER: oleg350-server.virtnet: executing load_modules_local [ 9838.909154] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 9838.943532] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 9840.171563] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 9840.193560] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 9840.257887] Lustre: lustre-MDT0000: new disk, initializing [ 9840.332697] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 9840.348739] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 9843.570558] Lustre: DEBUG MARKER: oleg350-server.virtnet: executing set_default_debug -1 all [ 9852.417896] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 9852.650717] Lustre: lustre-OST0000: new disk, initializing [ 9852.653976] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 9852.658029] Lustre: Skipped 1 previous similar message [ 9852.664746] Lustre: 312365:0:(osd_compat.c:1353:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 9852.756290] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 9854.134931] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:0:ost [ 9854.157260] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:0:ost] [ 9854.195441] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x240000400 [ 9858.908045] Lustre: DEBUG MARKER: oleg350-server.virtnet: executing set_default_debug -1 all [10765.796586] Lustre: lustre-MDT0000-lwp-OST0000: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [10765.811717] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [10769.124660] Lustre: server umount lustre-MDT0000 complete [10773.856168] LustreError: 311551:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1788649470 with bad export cookie 11463221948691903424 [10773.875239] LustreError: 311551:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 2 previous similar messages [10773.875758] LustreError: MGC192.168.203.150@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [10774.079834] Lustre: server umount lustre-OST0000 complete [10796.474209] Lustre: DEBUG MARKER: oleg350-server.virtnet: executing set_hostid [10805.041517] Lustre: DEBUG MARKER: oleg350-server.virtnet: executing load_modules_local [10833.804507] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [10834.025945] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [10834.066686] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [10834.125478] Lustre: lustre-MDT0000: new disk, initializing [10834.178842] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [10834.198731] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [10837.581081] Lustre: DEBUG MARKER: oleg350-server.virtnet: executing set_default_debug -1 all [10847.175824] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [10847.251739] Lustre: 316721:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [10847.293719] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [10847.296278] Lustre: Skipped 1 previous similar message [10847.353199] Lustre: lustre-MDT0001: new disk, initializing [10847.418866] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [10847.474786] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [10847.489300] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [10851.890137] Lustre: DEBUG MARKER: oleg350-server.virtnet: executing set_default_debug -1 all [10861.487436] Lustre: DEBUG MARKER: oleg350-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [10869.410198] Lustre: DEBUG MARKER: oleg350-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [10877.889534] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [10878.342262] Lustre: lustre-OST0000: new disk, initializing [10878.347758] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [10878.361567] Lustre: 318689:0:(osd_compat.c:1353:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [10878.524850] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [10885.291708] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [10885.304879] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [10885.397842] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [10887.358847] Lustre: DEBUG MARKER: oleg350-server.virtnet: executing set_default_debug -1 all [10900.142378] Lustre: DEBUG MARKER: oleg350-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [10904.703953] Lustre: DEBUG MARKER: oleg350-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 [10904.943770] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [10910.298955] Lustre: DEBUG MARKER: oleg350-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid 50 [10910.475548] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [10916.326361] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [10916.332613] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [10916.337077] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [10916.352788] Lustre: Skipped 1 previous similar message [10916.373419] Lustre: Skipped 1 previous similar message [10921.455529] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [10921.462847] Lustre: Skipped 1 previous similar message [10926.564886] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [10926.580861] Lustre: Skipped 1 previous similar message [10927.072050] Lustre: lustre-OST0000 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [10927.389282] Lustre: server umount lustre-OST0000 complete [10934.754814] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [10934.767298] LustreError: Skipped 1 previous similar message [10934.774364] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [10934.788705] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [10936.807312] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [10938.404918] Lustre: server umount lustre-MDT0000 complete [10941.839750] LustreError: 317513:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1788649638 with bad export cookie 11463221948694844243 [10941.842631] LustreError: MGC192.168.203.150@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [10941.850390] LustreError: 317513:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 2 previous similar messages [10942.135980] Lustre: server umount lustre-MDT0001 complete [10945.621815] Lustre: DEBUG MARKER: SKIP: conf-sanity test_111 skipping SLOW test 111 [10947.542394] Lustre: DEBUG MARKER: == conf-sanity test 112a: mount OST with no_create option ========================================================== 19:07:22 (1788649642) [10955.766839] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [10956.347505] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [10961.161454] Lustre: DEBUG MARKER: oleg350-server.virtnet: executing set_default_debug -1 all [10961.378252] LustreError: 321004:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [10961.403322] LustreError: 321004:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 2 previous similar messages [10966.501864] LustreError: 321003:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [10969.053980] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [10969.305233] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [10973.934841] Lustre: DEBUG MARKER: oleg350-server.virtnet: executing set_default_debug -1 all [10983.875442] Lustre: DEBUG MARKER: oleg350-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [10991.445416] Lustre: DEBUG MARKER: oleg350-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [11000.333752] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [11000.813982] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [11009.779615] Lustre: DEBUG MARKER: oleg350-server.virtnet: executing set_default_debug -1 all [11020.359523] Lustre: DEBUG MARKER: oleg350-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [11028.290558] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [11028.460369] Lustre: lustre-OST0001: new disk, initializing [11028.464281] Lustre: srv-lustre-OST0001: No data found on store. Initialize space. [11028.474402] Lustre: 323842:0:(osd_compat.c:1353:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [11028.556857] Lustre: lustre-OST0001: Imperative Recovery not enabled, recovery window 60-180 [11035.281235] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x00000002c0000400-0x0000000300000400]:1:ost [11035.303832] Lustre: cli-lustre-OST0001-super: Allocated super-sequence [0x00000002c0000400-0x0000000300000400]:1:ost] [11035.391285] LustreError: 324102:0:(osp_precreate.c:974:osp_precreate_cleanup_orphans()) lustre-OST0001-osc-MDT0001: cannot cleanup orphans: rc = -1 [11036.449287] LustreError: 324102:0:(osp_precreate.c:974:osp_precreate_cleanup_orphans()) lustre-OST0001-osc-MDT0001: cannot cleanup orphans: rc = -1 [11036.481406] LustreError: 324102:0:(osp_precreate.c:974:osp_precreate_cleanup_orphans()) Skipped 1 previous similar message [11037.545205] LustreError: 324102:0:(osp_precreate.c:974:osp_precreate_cleanup_orphans()) lustre-OST0001-osc-MDT0001: cannot cleanup orphans: rc = -1 [11037.555706] LustreError: 324102:0:(osp_precreate.c:974:osp_precreate_cleanup_orphans()) Skipped 1 previous similar message [11037.762905] Lustre: DEBUG MARKER: oleg350-server.virtnet: executing set_default_debug -1 all [11039.659193] LustreError: 324111:0:(osp_precreate.c:974:osp_precreate_cleanup_orphans()) lustre-OST0001-osc-MDT0000: cannot cleanup orphans: rc = -1 [11039.676882] LustreError: 324111:0:(osp_precreate.c:974:osp_precreate_cleanup_orphans()) Skipped 2 previous similar messages [11043.898323] LustreError: 324102:0:(osp_precreate.c:974:osp_precreate_cleanup_orphans()) lustre-OST0001-osc-MDT0001: cannot cleanup orphans: rc = -1 [11043.910316] LustreError: 324102:0:(osp_precreate.c:974:osp_precreate_cleanup_orphans()) Skipped 8 previous similar messages [11051.593402] Lustre: DEBUG MARKER: oleg350-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 [11051.940432] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [11052.197709] LustreError: 324102:0:(osp_precreate.c:974:osp_precreate_cleanup_orphans()) lustre-OST0001-osc-MDT0001: cannot cleanup orphans: rc = -1 [11052.208608] LustreError: 324102:0:(osp_precreate.c:974:osp_precreate_cleanup_orphans()) Skipped 15 previous similar messages [11058.437430] Lustre: DEBUG MARKER: oleg350-client.virtnet: executing wait_import_state (FULL|IDLE) osc.lustre-OST0000-osc-ffff922260943000.ost_server_uuid 50 [11059.952791] Lustre: DEBUG MARKER: osc.lustre-OST0000-osc-ffff922260943000.ost_server_uuid in FULL state after 0 sec [11065.209777] Lustre: DEBUG MARKER: oleg350-client.virtnet: executing wait_import_state (FULL|IDLE) osc.lustre-OST0001-osc-ffff922260943000.ost_server_uuid 50 [11066.613100] Lustre: DEBUG MARKER: osc.lustre-OST0001-osc-ffff922260943000.ost_server_uuid in FULL state after 0 sec [11068.652459] Lustre: lustre-OST0001-osc-MDT0000: update sequence from 0x100010000 to 0x2c0000401 [11084.262895] LustreError: lustre-OST0001-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [11084.264163] Lustre: lustre-OST0001-osc-MDT0001: Connection to lustre-OST0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [11084.277137] LustreError: Skipped 1 previous similar message [11084.286591] Lustre: lustre-OST0001: Not available for connect from 0@lo (stopping) [11084.311490] Lustre: Skipped 1 previous similar message [11084.339222] Lustre: Skipped 3 previous similar messages [11088.161259] Lustre: server umount lustre-OST0001 complete [11091.942731] LustreError: 325376:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [11091.959767] LustreError: 325376:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 1 previous similar message [11096.033625] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [11096.040412] LustreError: Skipped 1 previous similar message [11096.046416] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [11097.064853] LustreError: 322826:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [11097.097032] LustreError: 322826:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 2 previous similar messages [11100.479637] Lustre: server umount lustre-OST0000 complete [11107.298363] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [11107.309578] Lustre: Skipped 7 previous similar messages [11110.614336] Lustre: server umount lustre-MDT0000 complete [11112.430181] LustreError: 321003:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [11112.446459] LustreError: 321003:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 1 previous similar message [11114.261896] LustreError: 323841:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1788649811 with bad export cookie 11463221948694845125 [11114.266719] LustreError: MGC192.168.203.150@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [11114.275939] LustreError: 323841:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [11114.616801] Lustre: server umount lustre-MDT0001 complete [11126.130288] Lustre: DEBUG MARKER: oleg350-server.virtnet: executing unload_modules_local [11128.822665] Key type lgssc unregistered [11129.076836] LNet: 326532:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [11129.089821] LNetError: 326532:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [11129.127808] LNet: Removed LNI 192.168.203.150@tcp [11130.398670] Key type .llcrypt unregistered [11130.401102] Key type ._llcrypt unregistered [11157.048445] Key type ._llcrypt registered [11157.054115] Key type .llcrypt registered [11157.181600] Lustre: DEBUG MARKER: oleg350-server.virtnet: executing load_modules_local [11158.095488] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [11158.207472] alg: No test for adler32 (adler32-zlib) [11159.267864] Lustre: Lustre: Build Version: 2.17.58_3_g980e1e2 [11159.526598] LNet: Added LNI 192.168.203.150@tcp [8/256/0/180] [11161.207444] Key type lgssc registered [11162.327262] Lustre: Echo OBD driver; http://www.lustre.org/ [11170.267659] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [11170.281338] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [11171.666799] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [11176.104845] Lustre: DEBUG MARKER: oleg350-server.virtnet: executing set_default_debug -1 all [11182.050713] LustreError: 327955:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [11182.077519] LustreError: 327955:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 2 previous similar messages [11182.516736] Lustre: DEBUG MARKER: oleg350-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [11187.171784] LustreError: 327956:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [11188.967316] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [11189.390769] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [11193.598585] Lustre: DEBUG MARKER: oleg350-server.virtnet: executing set_default_debug -1 all [11204.134085] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [11204.681636] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [11211.007932] Lustre: DEBUG MARKER: oleg350-server.virtnet: executing set_default_debug -1 all [11214.949564] LustreError: 329635:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [11216.995160] LustreError: 329635:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [11217.023298] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:36 to 0x280000401:65) [11217.034277] LustreError: 329635:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 1 previous similar message [11219.348850] Lustre: DEBUG MARKER: oleg350-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [11222.508810] LustreError: 329636:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [11222.523741] LustreError: 329636:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 1 previous similar message [11226.652355] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [11226.878112] Lustre: lustre-OST0001: Imperative Recovery not enabled, recovery window 60-180 [11232.293778] Lustre: lustre-OST0001: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x2c0000401:3 to 0x2c0000401:33) [11235.032565] Lustre: DEBUG MARKER: oleg350-server.virtnet: executing set_default_debug -1 all [11243.488697] Lustre: DEBUG MARKER: oleg350-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0001-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [11252.939515] Lustre: DEBUG MARKER: oleg350-server.virtnet: executing wait_import_state (FULL|IDLE) os[cp].lustre-OST0001-osc-MDT0001.ost_server_uuid 50 [11253.118516] Lustre: DEBUG MARKER: os[cp].lustre-OST0001-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [11365.344435] LustreError: lustre-OST0001-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [11365.360344] Lustre: lustre-OST0001-osc-MDT0000: Connection to lustre-OST0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [11365.374710] Lustre: lustre-OST0001: Not available for connect from 0@lo (stopping) [11365.378366] Lustre: Skipped 1 previous similar message [11368.325231] Lustre: server umount lustre-OST0001 complete [11370.472834] LustreError: 329634:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [11370.499855] LustreError: 329634:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 1 previous similar message [11375.599096] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [11375.607816] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [11375.614352] Lustre: Skipped 2 previous similar messages [11375.638112] Lustre: Skipped 1 previous similar message [11380.709177] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [11380.716029] Lustre: Skipped 1 previous similar message [11381.368584] Lustre: server umount lustre-OST0000 complete [11385.827183] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [11385.846970] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [11385.851017] Lustre: Skipped 1 previous similar message [11390.944753] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [11390.954974] Lustre: Skipped 1 previous similar message [11391.638228] Lustre: server umount lustre-MDT0000 complete [11395.708842] LustreError: 327937:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1788650092 with bad export cookie 18047986785029128153 [11395.712878] LustreError: MGC192.168.203.150@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [11395.719975] LustreError: 327937:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 1 previous similar message [11396.224722] Lustre: server umount lustre-MDT0001 complete [11407.280939] Lustre: DEBUG MARKER: oleg350-server.virtnet: executing unload_modules_local [11410.351911] Key type lgssc unregistered [11410.650145] LNet: 332907:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [11410.663969] LNetError: 332907:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [11410.682282] LNet: Removed LNI 192.168.203.150@tcp [11411.641229] Key type .llcrypt unregistered [11411.647153] Key type ._llcrypt unregistered [11462.067928] Key type ._llcrypt registered [11462.070965] Key type .llcrypt registered [11462.205478] Lustre: DEBUG MARKER: oleg350-server.virtnet: executing load_modules_local [11463.939923] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [11464.016501] alg: No test for adler32 (adler32-zlib) [11465.297979] Lustre: Lustre: Build Version: 2.17.58_3_g980e1e2 [11465.747080] LNet: Added LNI 192.168.203.150@tcp [8/256/0/180] [11467.439195] Key type lgssc registered [11468.833867] Lustre: Echo OBD driver; http://www.lustre.org/ [11479.607857] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [11479.639770] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [11481.183855] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [11486.377581] Lustre: DEBUG MARKER: oleg350-server.virtnet: executing set_default_debug -1 all [11491.300848] LustreError: 335197:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [11491.313982] LustreError: 335197:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 2 previous similar messages [11495.666937] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [11496.095375] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [11500.464814] Lustre: DEBUG MARKER: oleg350-server.virtnet: executing set_default_debug -1 all [11503.073334] Lustre: Modifying parameter general.debug_raw_pointers=Y in log params [11510.556344] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [11510.973150] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [11517.451913] Lustre: DEBUG MARKER: oleg350-server.virtnet: executing set_default_debug -1 all [11520.164798] LustreError: 336685:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [11525.606463] LustreError: 336686:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [11526.658189] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:36 to 0x280000401:97) [11527.432568] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [11527.846577] Lustre: lustre-OST0001: Imperative Recovery not enabled, recovery window 60-180 [11533.315729] Lustre: lustre-OST0001: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x2c0000401:3 to 0x2c0000401:65) [11534.747835] Lustre: DEBUG MARKER: oleg350-server.virtnet: executing set_default_debug -1 all [11544.520650] Lustre: DEBUG MARKER: Using TIMEOUT=20 [11551.135193] Lustre: Setting parameter general.lod.*.mdt_hash=crush in log params [11564.002461] Lustre: lustre-MDT0000-lwp-OST0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [11564.009805] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [11564.025227] Lustre: Skipped 3 previous similar messages [11564.033785] Lustre: Skipped 3 previous similar messages [11569.121361] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [11569.142940] Lustre: Skipped 3 previous similar messages [11569.822887] Lustre: server umount lustre-MDT0000 complete [11573.955379] LustreError: 335179:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1788650270 with bad export cookie 14819580409066763879 [11573.963258] LustreError: MGC192.168.203.150@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [11573.981299] LustreError: 335179:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 3 previous similar messages [11574.269910] Lustre: server umount lustre-MDT0001 complete [11588.587498] Lustre: server umount lustre-OST0000 complete [11590.623225] Lustre: 334459:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1788650271/real 1788650271] req@ffff99a01768c380 x1875535632334592/t0(0) o400->lustre-MDT0001-lwp-OST0001@0@lo:12/10 lens 224/224 e 0 to 1 dl 1788650287 ref 1 fl Rpc:XNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 [11590.660946] Lustre: lustre-MDT0001-lwp-OST0001: Connection to lustre-MDT0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [11592.758040] Lustre: server umount lustre-OST0001 complete [11612.312404] Lustre: DEBUG MARKER: oleg350-server.virtnet: executing load_modules_local [11623.273396] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [11623.592364] LustreError: 340460:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [11623.601825] LustreError: 340460:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 2 previous similar messages [11623.630811] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [11628.443748] Lustre: DEBUG MARKER: oleg350-server.virtnet: executing set_default_debug -1 all [11629.024292] LustreError: 340461:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [11637.506558] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [11637.986918] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [11642.463159] Lustre: DEBUG MARKER: oleg350-server.virtnet: executing set_default_debug -1 all [11645.282885] Lustre: 341602:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify general/debug_raw_pointers=Y (mode = 0) failed: rc = -17 [11652.314369] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [11659.056860] Lustre: DEBUG MARKER: oleg350-server.virtnet: executing set_default_debug -1 all [11662.144761] LustreError: 341955:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [11662.159243] LustreError: 341955:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 1 previous similar message [11668.481368] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:36 to 0x280000401:129) [11669.008591] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [11669.265094] Lustre: lustre-OST0001: Imperative Recovery not enabled, recovery window 60-180 [11669.269694] Lustre: Skipped 1 previous similar message [11674.617547] Lustre: lustre-OST0001: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x2c0000401:3 to 0x2c0000401:97) [11675.046526] Lustre: DEBUG MARKER: oleg350-server.virtnet: executing set_default_debug -1 all [11683.035443] Lustre: DEBUG MARKER: Using TIMEOUT=20 [11686.529714] Lustre: 343471:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify general/lod.*.mdt_hash=crush (mode = 0) failed: rc = -17 [11694.048410] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [11694.068285] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [11694.089232] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [11697.377253] Lustre: server umount lustre-MDT0000 complete [11700.212308] LustreError: 340460:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [11700.240447] LustreError: 340460:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 6 previous similar messages [11701.201456] LustreError: 341604:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1788650398 with bad export cookie 14819580409066765580 [11701.203169] LustreError: MGC192.168.203.150@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [11701.214307] LustreError: 341604:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 1 previous similar message [11701.646572] Lustre: server umount lustre-MDT0001 complete [11716.118563] Lustre: server umount lustre-OST0000 complete [11730.604408] Lustre: server umount lustre-OST0001 complete [11734.823689] Lustre: DEBUG MARKER: SKIP: conf-sanity test_114 skipping SLOW test 114 [11736.681374] Lustre: DEBUG MARKER: SKIP: conf-sanity test_115 skipping excluded test 115 [11738.738084] Lustre: DEBUG MARKER: == conf-sanity test 116: big size MDT support ============ 19:20:34 (1788650434) [11763.683782] Lustre: DEBUG MARKER: oleg350-server.virtnet: executing load_modules_local [11773.075343] loop0: detected capacity change from 0 to 1099511627776 [11773.664207] SGI XFS with ACLs, security attributes, quota, no debug enabled [11773.672468] *** VALIDATE xfs *** [11773.688313] XFS (loop0): Mounting V5 Filesystem [11773.710248] XFS (loop0): Ending clean mount [11774.870635] loop1: detected capacity change from 0 to 18691697672192 [11885.335449] Lustre: DEBUG MARKER: oleg350-server.virtnet: executing set_hostid [11901.625600] Lustre: DEBUG MARKER: oleg350-server.virtnet: executing load_modules_local [11938.674961] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [11938.907804] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [11938.958433] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [11939.040165] Lustre: lustre-MDT0000: new disk, initializing [11939.113448] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [11939.140851] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [11943.531862] Lustre: DEBUG MARKER: oleg350-server.virtnet: executing set_default_debug -1 all [11953.341469] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [11953.460979] Lustre: 348635:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [11953.518617] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [11953.522841] Lustre: Skipped 1 previous similar message [11953.591447] Lustre: lustre-MDT0001: new disk, initializing [11953.692058] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [11953.704608] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [11958.013765] Lustre: DEBUG MARKER: oleg350-server.virtnet: executing set_default_debug -1 all [11967.229422] Lustre: DEBUG MARKER: oleg350-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [11973.462370] Lustre: DEBUG MARKER: oleg350-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [11980.423499] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [11980.712436] Lustre: lustre-OST0000: new disk, initializing [11980.715732] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [11980.727944] Lustre: 350603:0:(osd_compat.c:1353:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [11982.575900] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [11982.589986] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [11982.695186] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [11986.866323] Lustre: DEBUG MARKER: oleg350-server.virtnet: executing set_default_debug -1 all [11996.206984] Lustre: DEBUG MARKER: oleg350-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [12000.342861] Lustre: DEBUG MARKER: oleg350-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 [12000.516126] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [12004.042744] Lustre: DEBUG MARKER: oleg350-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid 50 [12004.294569] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [12008.423179] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [12008.424807] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [12008.425378] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [12008.431896] Lustre: Skipped 4 previous similar messages [12008.451458] Lustre: Skipped 5 previous similar messages [12012.321052] Lustre: server umount lustre-OST0000 complete [12018.658130] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [12018.667193] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [12018.681952] Lustre: Skipped 1 previous similar message [12018.698954] Lustre: Skipped 1 previous similar message [12022.159727] Lustre: server umount lustre-MDT0000 complete [12023.782213] LustreError: 348647:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [12023.796373] LustreError: 348647:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 1 previous similar message [12025.947548] LustreError: 348627:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1788650722 with bad export cookie 14819580409066767134 [12025.948622] LustreError: MGC192.168.203.150@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [12025.959123] LustreError: 348627:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 3 previous similar messages [12026.376027] Lustre: server umount lustre-MDT0001 complete [12034.966463] XFS (loop0): Unmounting Filesystem [12039.562676] Lustre: DEBUG MARKER: == conf-sanity test 117: lctl get_param return errors properly ========================================================== 19:25:35 (1788650735) [12045.694069] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [12046.314806] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [12046.319172] Lustre: Skipped 2 previous similar messages [12050.538911] Lustre: DEBUG MARKER: oleg350-server.virtnet: executing set_default_debug -1 all [12057.948427] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [12061.625732] Lustre: DEBUG MARKER: oleg350-server.virtnet: executing set_default_debug -1 all [12067.932563] Lustre: DEBUG MARKER: oleg350-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [12073.558557] Lustre: DEBUG MARKER: oleg350-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [12081.250335] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [12090.012834] Lustre: DEBUG MARKER: oleg350-server.virtnet: executing set_default_debug -1 all [12098.785836] Lustre: DEBUG MARKER: oleg350-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [12109.281907] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [12109.294686] LustreError: Skipped 1 previous similar message [12109.298632] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [12109.317319] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [12112.356375] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [12113.904371] Lustre: server umount lustre-OST0000 complete [12119.520211] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [12123.928614] Lustre: server umount lustre-MDT0000 complete [12127.565970] LustreError: 353139:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1788650824 with bad export cookie 14819580409066768016 [12127.576103] LustreError: MGC192.168.203.150@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [12127.579549] LustreError: 353139:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 3 previous similar messages [12127.988977] Lustre: server umount lustre-MDT0001 complete [12141.206379] Lustre: DEBUG MARKER: oleg350-server.virtnet: executing unload_modules_local [12143.984386] Key type lgssc unregistered [12144.311932] LNet: 356786:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [12144.319170] LNetError: 356786:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [12144.335866] LNet: Removed LNI 192.168.203.150@tcp [12145.146192] Key type .llcrypt unregistered [12145.147962] Key type ._llcrypt unregistered [12174.217747] Key type ._llcrypt registered [12174.222738] Key type .llcrypt registered [12174.335839] Lustre: DEBUG MARKER: oleg350-server.virtnet: executing load_modules_local [12175.364245] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [12175.396578] alg: No test for adler32 (adler32-zlib) [12176.484696] Lustre: Lustre: Build Version: 2.17.58_3_g980e1e2 [12176.728234] LNet: Added LNI 192.168.203.150@tcp [8/256/0/180] [12178.399368] Key type lgssc registered [12179.363711] Lustre: Echo OBD driver; http://www.lustre.org/ [12186.411363] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [12186.428687] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [12187.842625] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [12191.149620] Lustre: DEBUG MARKER: oleg350-server.virtnet: executing set_default_debug -1 all [12197.345360] LustreError: 358349:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [12197.357565] LustreError: 358349:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 2 previous similar messages [12199.507148] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [12199.978052] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [12204.629395] Lustre: DEBUG MARKER: oleg350-server.virtnet: executing set_default_debug -1 all [12211.782495] Lustre: DEBUG MARKER: oleg350-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [12217.382919] Lustre: DEBUG MARKER: oleg350-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [12224.133574] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [12224.590794] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [12230.675945] Lustre: DEBUG MARKER: oleg350-server.virtnet: executing set_default_debug -1 all [12238.294613] Lustre: DEBUG MARKER: oleg350-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [12245.991171] LustreError: lustre-MDT0001-osp-MDT0000: operation mds_statfs to node 0@lo failed: rc = -107 [12246.003746] Lustre: lustre-MDT0001-osp-MDT0000: Connection to lustre-MDT0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [12246.033798] Lustre: lustre-MDT0001: Not available for connect from 0@lo (stopping) [12248.035396] Lustre: lustre-MDT0001: Not available for connect from 0@lo (stopping) [12248.038249] Lustre: Skipped 1 previous similar message [12248.040073] Lustre: lustre-MDT0001-lwp-OST0000: Connection to lustre-MDT0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [12251.248212] Lustre: lustre-MDT0001: Not available for connect from 192.168.203.50@tcp (stopping) [12251.256523] Lustre: Skipped 1 previous similar message [12256.371041] Lustre: lustre-MDT0001: Not available for connect from 192.168.203.50@tcp (stopping) [12256.396126] Lustre: Skipped 2 previous similar messages [12260.319117] Lustre: lustre-MDT0001 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [12260.469851] Lustre: server umount lustre-MDT0001 complete [12261.485112] LustreError: 358344:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 192.168.203.50@tcp (no target). If you are running an HA pair check that the target is mounted on the other server. [12263.404344] LustreError: 358350:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [12263.420536] LustreError: 358350:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 1 previous similar message [12266.616538] LustreError: 358346:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 192.168.203.50@tcp (no target). If you are running an HA pair check that the target is mounted on the other server. [12271.429576] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [12271.537456] Lustre: MGS: Regenerating lustre-MDT0001 log by user request: rc = 0 [12271.540162] Lustre: Found index 1 for lustre-MDT0001, updating log [12271.601500] Lustre: 358337:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [12271.728934] Lustre: lustre-MDT0001: Not available for connect from 192.168.203.50@tcp (not set up) [12271.740586] Lustre: Skipped 2 previous similar messages [12271.921729] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [12277.257470] Lustre: lustre-MDT0001-lwp-OST0000: Connection restored to 0@lo (at 0@lo) [12277.469567] Lustre: DEBUG MARKER: oleg350-server.virtnet: executing set_default_debug -1 all [12279.283098] LustreError: 361880:0:(obd_mount.c:234:lustre_start_simple()) lustre-MDT0001-lwp-OST0000 attach error -17 [12279.306367] LustreError: 361880:0:(tgt_mount.c:675:lustre_lwp_setup()) lustre-MDT0001-lwp-OST0000: setup up failed: rc -17 [12279.326603] LustreError: 358341:0:(mgc_request.c:614:do_requeue()) failed processing log: -17 [12279.429398] LustreError: 361889:0:(obd_config.c:2102:class_config_llog_handler()) MGC192.168.203.150@tcp: cfg command failed: rc = -17 [12279.443539] Lustre: cmd=cf001 0:lustre-MDT0001-osp-MDT0000 1:osp 2:lustre-MDT0000-mdtlov_UUID [12279.443539] [12287.109063] Lustre: lustre-MDT0001: Not available for connect from 192.168.203.50@tcp (stopping) [12287.457768] Lustre: lustre-MDT0001-osp-MDT0000: Connection to lustre-MDT0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [12287.459171] LustreError: lustre-MDT0001-osp-MDT0000: operation mds_statfs to node 0@lo failed: rc = -107 [12301.279252] Lustre: lustre-MDT0001 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [12301.485725] Lustre: server umount lustre-MDT0001 complete [12302.445907] LustreError: 358344:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 192.168.203.50@tcp (no target). If you are running an HA pair check that the target is mounted on the other server. [12302.467530] LustreError: 358344:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 2 previous similar messages [12310.955259] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [12311.088556] Lustre: MGS: Regenerating lustre-MDT0001 log by user request: rc = 0 [12311.092062] Lustre: Found index 1 for lustre-MDT0001, updating log [12311.135293] Lustre: 358337:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [12311.517970] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [12315.806408] Lustre: DEBUG MARKER: oleg350-server.virtnet: executing set_default_debug -1 all [12316.657094] Lustre: lustre-MDT0001-lwp-OST0000: Connection restored to 0@lo (at 0@lo) [12318.210772] LustreError: 362999:0:(obd_mount.c:234:lustre_start_simple()) lustre-MDT0001-lwp-OST0000 attach error -17 [12318.221895] LustreError: 362999:0:(tgt_mount.c:675:lustre_lwp_setup()) lustre-MDT0001-lwp-OST0000: setup up failed: rc -17 [12318.234264] LustreError: 358341:0:(mgc_request.c:614:do_requeue()) failed processing log: -17 [12318.241197] LustreError: 358341:0:(mgc_request.c:614:do_requeue()) Skipped 1 previous similar message [12318.299221] LustreError: 363004:0:(obd_config.c:761:class_setup()) osp: device 8 already setup: rc = -17 [12318.308677] LustreError: 363004:0:(obd_config.c:866:class_detach()) OBD device 8 still set up [12318.314752] LustreError: 363004:0:(obd_config.c:2102:class_config_llog_handler()) MGC192.168.203.150@tcp: cfg command failed: rc = -17 [12318.328343] Lustre: cmd=cf003 0:lustre-MDT0001-osp-MDT0000 1:lustre-MDT0001_UUID 2:192.168.203.150@tcp [12318.328343] [12621.798780] Lustre: lustre-MDT0001-lwp-OST0000: Connection to lustre-MDT0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [12621.802366] Lustre: lustre-MDT0001: Not available for connect from 0@lo (stopping) [12621.807602] Lustre: Skipped 2 previous similar messages [12621.818158] Lustre: Skipped 9 previous similar messages [12627.493938] Lustre: server umount lustre-MDT0001 complete [12630.128239] LustreError: 358345:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 192.168.203.50@tcp (no target). If you are running an HA pair check that the target is mounted on the other server. [12630.146950] LustreError: 358345:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 5 previous similar messages [12638.958950] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [12639.056332] Lustre: MGS: Regenerating lustre-MDT0001 log by user request: rc = 0 [12639.058689] Lustre: Found index 1 for lustre-MDT0001, updating log [12639.085231] Lustre: 358338:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [12639.457962] Lustre: lustre-MDT0001: Imperative Recovery enabled, recovery window shrunk from 60-180 down to 60-180 [12644.844796] Lustre: lustre-MDT0001-lwp-OST0000: Connection restored to 0@lo (at 0@lo) [12645.004113] Lustre: DEBUG MARKER: oleg350-server.virtnet: executing set_default_debug -1 all [12648.960278] LustreError: 372201:0:(obd_mount.c:234:lustre_start_simple()) lustre-MDT0001-lwp-OST0000 attach error -17 [12648.973797] LustreError: 372201:0:(tgt_mount.c:675:lustre_lwp_setup()) lustre-MDT0001-lwp-OST0000: setup up failed: rc -17 [12648.989989] LustreError: 358341:0:(mgc_request.c:614:do_requeue()) failed processing log: -17 [12649.003282] LustreError: 358341:0:(mgc_request.c:614:do_requeue()) Skipped 1 previous similar message [12649.095502] LustreError: 372204:0:(lod_lov.c:208:lod_add_device()) lustre-MDT0001-osp-MDT0000: cannot connect to next dev lustre-MDT0001_UUID (-114) [12649.125058] LustreError: 372204:0:(obd_config.c:2102:class_config_llog_handler()) MGC192.168.203.150@tcp: cfg command failed: rc = -114 [12649.152130] Lustre: cmd=cf014 0:lustre-MDT0000-mdtlov 1:lustre-MDT0001_UUID 2:1 3:1 [12649.152130] [12967.392801] Lustre: lustre-MDT0000-lwp-OST0000: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [12967.393457] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [12967.394645] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [12967.416664] Lustre: Skipped 2 previous similar messages [12967.424776] Lustre: Skipped 4 previous similar messages [12973.033472] Lustre: server umount lustre-MDT0000 complete [12976.686217] LustreError: 358331:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1788651673 with bad export cookie 5148886376883899214 [12976.702553] LustreError: MGC192.168.203.150@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [12976.706942] LustreError: 358331:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 2 previous similar messages [12977.633094] LustreError: 358345:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [12977.633357] Lustre: lustre-MDT0001-lwp-OST0000: Connection to lustre-MDT0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [12977.654295] LustreError: 358345:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 5 previous similar messages [12983.050517] Lustre: server umount lustre-MDT0001 complete [12987.245533] Lustre: server umount lustre-OST0000 complete [12996.557655] Lustre: DEBUG MARKER: oleg350-server.virtnet: executing set_hostid [13007.644324] Lustre: DEBUG MARKER: oleg350-server.virtnet: executing load_modules_local [13047.754444] Lustre: DEBUG MARKER: == conf-sanity test 120: cross-target rename should not create bad symlinks ========================================================== 19:42:23 (1788651743) [13054.802863] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [13055.185831] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [13055.283505] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [13055.374512] Lustre: lustre-MDT0000: new disk, initializing [13055.548362] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [13055.579450] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [13060.363362] Lustre: DEBUG MARKER: oleg350-server.virtnet: executing set_default_debug -1 all [13071.128614] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [13071.272607] Lustre: 383195:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [13071.343074] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [13071.346599] Lustre: Skipped 1 previous similar message [13071.469619] Lustre: lustre-MDT0001: new disk, initializing [13071.574280] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [13071.606490] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [13071.629791] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [13076.986293] Lustre: DEBUG MARKER: oleg350-server.virtnet: executing set_default_debug -1 all [13087.066567] Lustre: DEBUG MARKER: oleg350-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [13092.995695] Lustre: DEBUG MARKER: oleg350-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [13100.136640] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [13100.495667] Lustre: lustre-OST0000: new disk, initializing [13100.502958] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [13100.515918] Lustre: 385163:0:(osd_compat.c:1353:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [13100.640793] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [13108.072889] Lustre: DEBUG MARKER: oleg350-server.virtnet: executing set_default_debug -1 all [13109.856106] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [13109.875535] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [13110.004121] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [13117.966844] Lustre: DEBUG MARKER: oleg350-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [13127.065329] LustreError: lustre-MDT0000-osp-MDT0001: operation out_update to node 0@lo failed: rc = -107 [13127.085824] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [13127.119774] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [13127.133719] Lustre: Skipped 6 previous similar messages [13127.349201] Lustre: server umount lustre-MDT0000 complete [13130.735733] LustreError: 383205:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [13130.768917] LustreError: 383205:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 1 previous similar message [13131.765963] LustreError: 383185:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1788651828 with bad export cookie 5148886376883901167 [13131.770508] LustreError: MGC192.168.203.150@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [13131.771547] LustreError: 384001:0:(osp_object.c:618:osp_attr_get()) lustre-MDT0000-osp-MDT0001: osp_attr_get update error [0x200000401:0x2:0x0]: rc = -5 [13131.774326] LustreError: 383185:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 1 previous similar message [13132.073312] Lustre: server umount lustre-MDT0001 complete [13146.288976] Lustre: server umount lustre-OST0000 complete [13177.373923] Lustre: DEBUG MARKER: oleg350-server.virtnet: executing set_hostid [13188.136911] Lustre: DEBUG MARKER: oleg350-server.virtnet: executing load_modules_local [13222.500268] Lustre: DEBUG MARKER: == conf-sanity test complete, duration 12932 sec ========= 19:45:17 (1788651917) [13224.305071] Lustre: DEBUG MARKER: === conf-sanity: start cleanup 19:45:19 (1788651919) === [13227.784763] Lustre: DEBUG MARKER: === conf-sanity: finish cleanup 19:45:23 (1788651923) === [13265.615060] Lustre: DEBUG MARKER: oleg350-server.virtnet: executing unload_modules_local [13268.386820] Key type lgssc unregistered [13268.692599] LNet: 391530:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [13268.699992] LNetError: 391530:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [13268.712285] LNet: Removed LNI 192.168.203.150@tcp [13269.599770] Key type .llcrypt unregistered [13269.602766] Key type ._llcrypt unregistered