[ 0.000000] Linux version 4.18.0rh8.10-debug (green@maintenance) (gcc version 8.5.0 20210514 (Red Hat 8.5.0-26) (GCC)) #2 SMP Mon Jul 14 01:24:22 EDT 2025 [ 0.000000] Command line: rd.shell root=nbd:192.168.200.253:rocky8.10:ext4:ro:-p,-b4096 ro crashkernel=256M panic=1 nomodeset ipmtu=9000 ip=dhcp rd.neednet=1 init_on_free=off mitigations=off console=ttyS1,115200 audit=0 [ 0.000000] x86/fpu: Supporting XSAVE feature 0x001: 'x87 floating point registers' [ 0.000000] x86/fpu: Supporting XSAVE feature 0x002: 'SSE registers' [ 0.000000] x86/fpu: Supporting XSAVE feature 0x004: 'AVX registers' [ 0.000000] x86/fpu: xstate_offset[2]: 576, xstate_sizes[2]: 256 [ 0.000000] x86/fpu: Enabled xstate features 0x7, context size is 832 bytes, using 'standard' format. [ 0.000000] signal: max sigframe size: 1776 [ 0.000000] BIOS-provided physical RAM map: [ 0.000000] BIOS-e820: [mem 0x0000000000000000-0x000000000009fbff] usable [ 0.000000] BIOS-e820: [mem 0x000000000009fc00-0x000000000009ffff] reserved [ 0.000000] BIOS-e820: [mem 0x00000000000f0000-0x00000000000fffff] reserved [ 0.000000] BIOS-e820: [mem 0x0000000000100000-0x00000000bffcdfff] usable [ 0.000000] BIOS-e820: [mem 0x00000000bffce000-0x00000000bfffffff] reserved [ 0.000000] BIOS-e820: [mem 0x00000000feffc000-0x00000000feffffff] reserved [ 0.000000] BIOS-e820: [mem 0x00000000fffc0000-0x00000000ffffffff] reserved [ 0.000000] BIOS-e820: [mem 0x0000000100000000-0x0000000146dfffff] usable [ 0.000000] NX (Execute Disable) protection: active [ 0.000000] SMBIOS 2.8 present. [ 0.000000] DMI: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.17.0-10.fc44 06/10/2025 [ 0.000000] Hypervisor detected: KVM [ 0.000000] kvm-clock: Using msrs 4b564d01 and 4b564d00 [ 0.000000] kvm-clock: using sched offset of 469728176 cycles [ 0.000000] clocksource: kvm-clock: mask: 0xffffffffffffffff max_cycles: 0x1cd42e4dffb, max_idle_ns: 881590591483 ns [ 0.000000] tsc: Detected 2399.998 MHz processor [ 0.000000] last_pfn = 0x146e00 max_arch_pfn = 0x400000000 [ 0.000000] x86/PAT: Configuration [0-7]: WB WC UC- UC WB WP UC- WT [ 0.000000] last_pfn = 0xbffce max_arch_pfn = 0x400000000 [ 0.000000] found SMP MP-table at [mem 0x000f54b0-0x000f54bf] [ 0.000000] RAMDISK: [mem 0xbcc54000-0xbffbffff] [ 0.000000] ACPI: Early table checksum verification disabled [ 0.000000] ACPI: RSDP 0x00000000000F52D0 000014 (v00 BOCHS ) [ 0.000000] ACPI: RSDT 0x00000000BFFE247C 000034 (v01 BOCHS BXPC 00000001 BXPC 00000001) [ 0.000000] ACPI: FACP 0x00000000BFFE2318 000074 (v01 BOCHS BXPC 00000001 BXPC 00000001) [ 0.000000] ACPI: DSDT 0x00000000BFFE0040 0022D8 (v01 BOCHS BXPC 00000001 BXPC 00000001) [ 0.000000] ACPI: FACS 0x00000000BFFE0000 000040 [ 0.000000] ACPI: APIC 0x00000000BFFE238C 000090 (v03 BOCHS BXPC 00000001 BXPC 00000001) [ 0.000000] ACPI: HPET 0x00000000BFFE241C 000038 (v01 BOCHS BXPC 00000001 BXPC 00000001) [ 0.000000] ACPI: WAET 0x00000000BFFE2454 000028 (v01 BOCHS BXPC 00000001 BXPC 00000001) [ 0.000000] ACPI: Reserving FACP table memory at [mem 0xbffe2318-0xbffe238b] [ 0.000000] ACPI: Reserving DSDT table memory at [mem 0xbffe0040-0xbffe2317] [ 0.000000] ACPI: Reserving FACS table memory at [mem 0xbffe0000-0xbffe003f] [ 0.000000] ACPI: Reserving APIC table memory at [mem 0xbffe238c-0xbffe241b] [ 0.000000] ACPI: Reserving HPET table memory at [mem 0xbffe241c-0xbffe2453] [ 0.000000] ACPI: Reserving WAET table memory at [mem 0xbffe2454-0xbffe247b] [ 0.000000] No NUMA configuration found [ 0.000000] Faking a node at [mem 0x0000000000000000-0x0000000146dfffff] [ 0.000000] NODE_DATA(0) allocated [mem 0x1465a3000-0x1465cdfff] [ 0.000000] Reserving 256MB of memory at 2656MB for crashkernel (System RAM: 4205MB) [ 0.000000] Zone ranges: [ 0.000000] DMA [mem 0x0000000000001000-0x0000000000ffffff] [ 0.000000] DMA32 [mem 0x0000000001000000-0x00000000ffffffff] [ 0.000000] Normal [mem 0x0000000100000000-0x0000000146dfffff] [ 0.000000] Device empty [ 0.000000] Movable zone start for each node [ 0.000000] Early memory node ranges [ 0.000000] node 0: [mem 0x0000000000001000-0x000000000009efff] [ 0.000000] node 0: [mem 0x0000000000100000-0x00000000bffcdfff] [ 0.000000] node 0: [mem 0x0000000100000000-0x0000000146dfffff] [ 0.000000] Zeroed struct page in unavailable ranges: 4756 pages [ 0.000000] Initmem setup node 0 [mem 0x0000000000001000-0x0000000146dfffff] [ 0.000000] ACPI: PM-Timer IO Port: 0x608 [ 0.000000] ACPI: LAPIC_NMI (acpi_id[0xff] dfl dfl lint[0x1]) [ 0.000000] IOAPIC[0]: apic_id 0, version 17, address 0xfec00000, GSI 0-23 [ 0.000000] ACPI: INT_SRC_OVR (bus 0 bus_irq 0 global_irq 2 dfl dfl) [ 0.000000] ACPI: INT_SRC_OVR (bus 0 bus_irq 5 global_irq 5 high level) [ 0.000000] ACPI: INT_SRC_OVR (bus 0 bus_irq 9 global_irq 9 high level) [ 0.000000] ACPI: INT_SRC_OVR (bus 0 bus_irq 10 global_irq 10 high level) [ 0.000000] ACPI: INT_SRC_OVR (bus 0 bus_irq 11 global_irq 11 high level) [ 0.000000] Using ACPI (MADT) for SMP configuration information [ 0.000000] ACPI: HPET id: 0x8086a201 base: 0xfed00000 [ 0.000000] TSC deadline timer available [ 0.000000] smpboot: Allowing 4 CPUs, 0 hotplug CPUs [ 0.000000] kvm-guest: KVM setup pv remote TLB flush [ 0.000000] kvm-guest: setup PV sched yield [ 0.000000] PM: Registered nosave memory: [mem 0x00000000-0x00000fff] [ 0.000000] PM: Registered nosave memory: [mem 0x0009f000-0x0009ffff] [ 0.000000] PM: Registered nosave memory: [mem 0x000a0000-0x000effff] [ 0.000000] PM: Registered nosave memory: [mem 0x000f0000-0x000fffff] [ 0.000000] PM: Registered nosave memory: [mem 0xbffce000-0xbfffffff] [ 0.000000] PM: Registered nosave memory: [mem 0xc0000000-0xfeffbfff] [ 0.000000] PM: Registered nosave memory: [mem 0xfeffc000-0xfeffffff] [ 0.000000] PM: Registered nosave memory: [mem 0xff000000-0xfffbffff] [ 0.000000] PM: Registered nosave memory: [mem 0xfffc0000-0xffffffff] [ 0.000000] [mem 0xc0000000-0xfeffbfff] available for PCI devices [ 0.000000] Booting paravirtualized kernel on KVM [ 0.000000] clocksource: refined-jiffies: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 1910969940391419 ns [ 0.000000] setup_percpu: NR_CPUS:8192 nr_cpumask_bits:4 nr_cpu_ids:4 nr_node_ids:1 [ 0.000000] percpu: Embedded 63 pages/cpu s221184 r8192 d28672 u524288 [ 0.000000] kvm-guest: PV spinlocks enabled [ 0.000000] PV qspinlock hash table entries: 256 (order: 0, 4096 bytes, linear) [ 0.000000] Built 1 zonelists, mobility grouping on. Total pages: 1059606 [ 0.000000] Policy zone: Normal [ 0.000000] Kernel command line: rd.shell root=nbd:192.168.200.253:rocky8.10:ext4:ro:-p,-b4096 ro crashkernel=256M panic=1 nomodeset ipmtu=9000 ip=dhcp rd.neednet=1 init_on_free=off mitigations=off console=ttyS1,115200 audit=0 [ 0.000000] Specific versions of hardware are certified with Red Hat Enterprise Linux 8. Please see the list of hardware certified with Red Hat Enterprise Linux 8 at https://catalog.redhat.com. [ 0.000000] audit: disabled (until reboot) [ 0.000000] software IO TLB: area num 4. [ 0.000000] Memory: 2829652K/4306352K available (18435K kernel code, 11221K rwdata, 7248K rodata, 2908K init, 18040K bss, 524584K reserved, 0K cma-reserved) [ 0.000000] SLUB: HWalign=64, Order=0-3, MinObjects=0, CPUs=4, Nodes=1 [ 0.000000] kmemleak: Kernel memory leak detector disabled [ 0.000000] ftrace: allocating 41240 entries in 162 pages [ 0.000000] ftrace: allocated 162 pages with 3 groups [ 0.000000] rcu: Hierarchical RCU implementation. [ 0.000000] rcu: RCU event tracing is enabled. [ 0.000000] rcu: RCU restricting CPUs from NR_CPUS=8192 to nr_cpu_ids=4. [ 0.000000] rcu: RCU callback double-/use-after-free debug enabled. [ 0.000000] Rude variant of Tasks RCU enabled. [ 0.000000] Tracing variant of Tasks RCU enabled. [ 0.000000] rcu: RCU calculated value of scheduler-enlistment delay is 100 jiffies. [ 0.000000] rcu: Adjusting geometry for rcu_fanout_leaf=16, nr_cpu_ids=4 [ 0.000000] NR_IRQS: 524544, nr_irqs: 456, preallocated irqs: 16 [ 0.000000] random: get_random_bytes called from start_kernel+0x622/0x9a8 with crng_init=0 [ 0.001000] Console: colour *CGA 80x25 [ 0.001000] printk: console [ttyS1] enabled [ 0.001000] ACPI: Core revision 20220331 [ 0.001000] clocksource: hpet: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 19112604467 ns [ 0.001014] APIC: Switch to symmetric I/O mode setup [ 0.003331] x2apic enabled [ 0.004008] Switched APIC routing to physical x2apic. [ 0.005016] kvm-guest: setup PV IPIs [ 0.008000] ..TIMER: vector=0x30 apic1=0 pin1=2 apic2=-1 pin2=-1 [ 0.008000] clocksource: tsc-early: mask: 0xffffffffffffffff max_cycles: 0x229835b7123, max_idle_ns: 440795242976 ns [ 0.008016] Calibrating delay loop (skipped) preset value.. 4799.99 BogoMIPS (lpj=2399998) [ 0.009012] pid_max: default: 32768 minimum: 301 [ 0.010126] LSM: Security Framework initializing [ 0.011058] Yama: becoming mindful. [ 0.012040] SELinux: Initializing. [ 0.013066] *** VALIDATE selinux *** [ 0.021376] Dentry cache hash table entries: 1048576 (order: 11, 8388608 bytes, vmalloc) [ 0.026645] Inode-cache hash table entries: 524288 (order: 10, 4194304 bytes, vmalloc) [ 0.028064] Mount-cache hash table entries: 16384 (order: 5, 131072 bytes, vmalloc) [ 0.030112] Mountpoint-cache hash table entries: 16384 (order: 5, 131072 bytes, vmalloc) [ 0.031119] *** VALIDATE tmpfs *** [ 0.033304] *** VALIDATE proc *** [ 0.035045] *** VALIDATE cgroup *** [ 0.036009] *** VALIDATE cgroup2 *** [ 0.038111] x86/cpu: User Mode Instruction Prevention (UMIP) activated [ 0.039153] Last level iTLB entries: 4KB 0, 2MB 0, 4MB 0 [ 0.040008] Last level dTLB entries: 4KB 0, 2MB 0, 4MB 0, 1GB 0 [ 0.042012] Spectre V2 : User space: Vulnerable [ 0.043010] Speculative Store Bypass: Vulnerable [ 0.046224] debug: unmapping init [mem 0xffffffff90a59000-0xffffffff90a60fff] [ 0.048181] smpboot: CPU0: Intel(R) Xeon(R) CPU E5-2695 v2 @ 2.40GHz (family: 0x6, model: 0x3e, stepping: 0x4) [ 0.049725] Performance Events: IvyBridge events, full-width counters, Intel PMU driver. [ 0.050023] ... version: 2 [ 0.051013] ... bit width: 48 [ 0.052012] ... generic registers: 4 [ 0.053013] ... value mask: 0000ffffffffffff [ 0.054016] ... max period: 00007fffffffffff [ 0.055014] ... fixed-purpose events: 3 [ 0.056009] ... event mask: 000000070000000f [ 0.057303] rcu: Hierarchical SRCU implementation. [ 0.059354] smp: Bringing up secondary CPUs ... [ 0.060593] x86: Booting SMP configuration: [ 0.061025] .... node #0, CPUs: #1 #2 #3 [ 0.064374] smp: Brought up 1 node, 4 CPUs [ 0.066014] smpboot: Max logical packages: 1 [ 0.067018] smpboot: Total of 4 processors activated (19199.98 BogoMIPS) [ 0.149252] node 0 deferred pages initialised in 80ms [ 0.152189] devtmpfs: initialized [ 0.154311] x86/mm: Memory block size: 128MB [ 0.156247] gcov: version magic: 0x41383552 [ 0.159167] clocksource: jiffies: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 1911260446275000 ns [ 0.162118] futex hash table entries: 1024 (order: 4, 65536 bytes, vmalloc) [ 0.166341] pinctrl core: initialized pinctrl subsystem [ 0.167145] [ 0.167692] ************************************************************* [ 0.170011] ** NOTICE NOTICE NOTICE NOTICE NOTICE NOTICE NOTICE ** [ 0.173012] ** ** [ 0.175010] ** IOMMU DebugFS SUPPORT HAS BEEN ENABLED IN THIS KERNEL ** [ 0.178011] ** ** [ 0.181010] ** This means that this kernel is built to expose internal ** [ 0.184018] ** IOMMU data structures, which may compromise security on ** [ 0.186007] ** your system. ** [ 0.187009] ** ** [ 0.190011] ** If you see this message and you are not debugging the ** [ 0.192011] ** kernel, report this immediately to your vendor! ** [ 0.195022] ** ** [ 0.197009] ** NOTICE NOTICE NOTICE NOTICE NOTICE NOTICE NOTICE ** [ 0.199010] ************************************************************* [ 0.201613] NET: Registered protocol family 16 [ 0.203441] DMA: preallocated 512 KiB GFP_KERNEL pool for atomic allocations [ 0.209064] DMA: preallocated 512 KiB GFP_KERNEL|GFP_DMA pool for atomic allocations [ 0.212066] DMA: preallocated 512 KiB GFP_KERNEL|GFP_DMA32 pool for atomic allocations [ 0.216033] cpuidle: using governor menu [ 0.217829] acpiphp: ACPI Hot Plug PCI Controller Driver version: 0.5 [ 0.220449] PCI: Using configuration type 1 for base access [ 0.222124] core: PMU erratum BJ122, BV98, HSD29 worked around, HT is on [ 0.233019] HugeTLB registered 1.00 GiB page size, pre-allocated 0 pages [ 0.234020] HugeTLB registered 2.00 MiB page size, pre-allocated 0 pages [ 0.235169] cryptd: max_cpu_qlen set to 1000 [ 0.239232] ACPI: Added _OSI(Module Device) [ 0.241014] ACPI: Added _OSI(Processor Device) [ 0.243013] ACPI: Added _OSI(3.0 _SCP Extensions) [ 0.244010] ACPI: Added _OSI(Processor Aggregator Device) [ 0.248952] ACPI: 1 ACPI AML tables successfully acquired and loaded [ 0.256633] ACPI: Interpreter enabled [ 0.258063] ACPI: PM: (supports S0 S3 S4 S5) [ 0.260015] ACPI: Using IOAPIC for interrupt routing [ 0.262097] PCI: Using host bridge windows from ACPI; if necessary, use "pci=nocrs" and report a bug [ 0.266406] ACPI: Enabled 2 GPEs in block 00 to 0F [ 0.275300] ACPI: PCI Root Bridge [PCI0] (domain 0000 [bus 00-ff]) [ 0.280041] acpi PNP0A03:00: _OSC: OS supports [ASPM ClockPM Segments MSI HPX-Type3] [ 0.282015] acpi PNP0A03:00: _OSC: not requesting OS control; OS requires [ExtendedConfig ASPM ClockPM MSI] [ 0.286080] acpi PNP0A03:00: fail to add MMCONFIG information, can't access extended PCI configuration space under this bridge. [ 0.291385] acpiphp: Slot [2] registered [ 0.293117] acpiphp: Slot [5] registered [ 0.295104] acpiphp: Slot [6] registered [ 0.296127] acpiphp: Slot [7] registered [ 0.297070] acpiphp: Slot [8] registered [ 0.299185] acpiphp: Slot [9] registered [ 0.301121] acpiphp: Slot [10] registered [ 0.303100] acpiphp: Slot [3] registered [ 0.304093] acpiphp: Slot [4] registered [ 0.306095] acpiphp: Slot [11] registered [ 0.308101] acpiphp: Slot [12] registered [ 0.309120] acpiphp: Slot [13] registered [ 0.311111] acpiphp: Slot [14] registered [ 0.313106] acpiphp: Slot [15] registered [ 0.315106] acpiphp: Slot [16] registered [ 0.316102] acpiphp: Slot [17] registered [ 0.318098] acpiphp: Slot [18] registered [ 0.320103] acpiphp: Slot [19] registered [ 0.321122] acpiphp: Slot [20] registered [ 0.323102] acpiphp: Slot [21] registered [ 0.325133] acpiphp: Slot [22] registered [ 0.327126] acpiphp: Slot [23] registered [ 0.328086] acpiphp: Slot [24] registered [ 0.330094] acpiphp: Slot [25] registered [ 0.331109] acpiphp: Slot [26] registered [ 0.333086] acpiphp: Slot [27] registered [ 0.334071] acpiphp: Slot [28] registered [ 0.335096] acpiphp: Slot [29] registered [ 0.336069] acpiphp: Slot [30] registered [ 0.338098] acpiphp: Slot [31] registered [ 0.340084] PCI host bridge to bus 0000:00 [ 0.341017] pci_bus 0000:00: root bus resource [io 0x0000-0x0cf7 window] [ 0.344022] pci_bus 0000:00: root bus resource [io 0x0d00-0xffff window] [ 0.346020] pci_bus 0000:00: root bus resource [mem 0x000a0000-0x000bffff window] [ 0.349024] pci_bus 0000:00: root bus resource [mem 0xc0000000-0xfebfffff window] [ 0.352023] pci_bus 0000:00: root bus resource [mem 0xe0000000000-0xe007fffffff window] [ 0.355024] pci_bus 0000:00: root bus resource [bus 00-ff] [ 0.357176] pci 0000:00:00.0: [8086:1237] type 00 class 0x060000 [ 0.360010] pci 0000:00:01.0: [8086:7000] type 00 class 0x060100 [ 0.363712] pci 0000:00:01.1: [8086:7010] type 00 class 0x010180 [ 0.374787] pci 0000:00:01.1: reg 0x20: [io 0xc320-0xc32f] [ 0.380567] pci 0000:00:01.1: legacy IDE quirk: reg 0x10: [io 0x01f0-0x01f7] [ 0.383018] pci 0000:00:01.1: legacy IDE quirk: reg 0x14: [io 0x03f6] [ 0.386019] pci 0000:00:01.1: legacy IDE quirk: reg 0x18: [io 0x0170-0x0177] [ 0.389018] pci 0000:00:01.1: legacy IDE quirk: reg 0x1c: [io 0x0376] [ 0.393568] pci 0000:00:01.3: [8086:7113] type 00 class 0x068000 [ 0.397082] pci 0000:00:01.3: quirk: [io 0x0600-0x063f] claimed by PIIX4 ACPI [ 0.400050] pci 0000:00:01.3: quirk: [io 0x0700-0x070f] claimed by PIIX4 SMB [ 0.402831] pci 0000:00:02.0: [1af4:1000] type 00 class 0x020000 [ 0.409016] pci 0000:00:02.0: reg 0x10: [io 0xc300-0xc31f] [ 0.422016] pci 0000:00:02.0: reg 0x20: [mem 0xe0000000000-0xe0000003fff 64bit pref] [ 0.429013] pci 0000:00:02.0: reg 0x30: [mem 0xfeb80000-0xfebbffff pref] [ 0.433873] pci 0000:00:05.0: [1af4:1001] type 00 class 0x010000 [ 0.443016] pci 0000:00:05.0: reg 0x10: [io 0xc000-0xc07f] [ 0.452014] pci 0000:00:05.0: reg 0x14: [mem 0xfebc0000-0xfebc0fff] [ 0.475017] pci 0000:00:05.0: reg 0x20: [mem 0xe0000004000-0xe0000007fff 64bit pref] [ 0.488012] pci 0000:00:06.0: [1af4:1001] type 00 class 0x010000 [ 0.500016] pci 0000:00:06.0: reg 0x10: [io 0xc080-0xc0ff] [ 0.512012] pci 0000:00:06.0: reg 0x14: [mem 0xfebc1000-0xfebc1fff] [ 0.532021] pci 0000:00:06.0: reg 0x20: [mem 0xe0000008000-0xe000000bfff 64bit pref] [ 0.543919] pci 0000:00:07.0: [1af4:1001] type 00 class 0x010000 [ 0.550015] pci 0000:00:07.0: reg 0x10: [io 0xc100-0xc17f] [ 0.556015] pci 0000:00:07.0: reg 0x14: [mem 0xfebc2000-0xfebc2fff] [ 0.569014] pci 0000:00:07.0: reg 0x20: [mem 0xe000000c000-0xe000000ffff 64bit pref] [ 0.579824] pci 0000:00:08.0: [1af4:1001] type 00 class 0x010000 [ 0.587015] pci 0000:00:08.0: reg 0x10: [io 0xc180-0xc1ff] [ 0.594016] pci 0000:00:08.0: reg 0x14: [mem 0xfebc3000-0xfebc3fff] [ 0.615011] pci 0000:00:08.0: reg 0x20: [mem 0xe0000010000-0xe0000013fff 64bit pref] [ 0.626487] pci 0000:00:09.0: [1af4:1001] type 00 class 0x010000 [ 0.633014] pci 0000:00:09.0: reg 0x10: [io 0xc200-0xc27f] [ 0.641014] pci 0000:00:09.0: reg 0x14: [mem 0xfebc4000-0xfebc4fff] [ 0.655061] pci 0000:00:09.0: reg 0x20: [mem 0xe0000014000-0xe0000017fff 64bit pref] [ 0.662607] pci 0000:00:0a.0: [1af4:1001] type 00 class 0x010000 [ 0.668014] pci 0000:00:0a.0: reg 0x10: [io 0xc280-0xc2ff] [ 0.673015] pci 0000:00:0a.0: reg 0x14: [mem 0xfebc5000-0xfebc5fff] [ 0.687020] pci 0000:00:0a.0: reg 0x20: [mem 0xe0000018000-0xe000001bfff 64bit pref] [ 0.697939] ACPI: PCI: Interrupt link LNKA configured for IRQ 10 [ 0.701397] ACPI: PCI: Interrupt link LNKB configured for IRQ 10 [ 0.703452] ACPI: PCI: Interrupt link LNKC configured for IRQ 11 [ 0.707399] ACPI: PCI: Interrupt link LNKD configured for IRQ 11 [ 0.710264] ACPI: PCI: Interrupt link LNKS configured for IRQ 9 [ 0.714181] iommu: Default domain type: Passthrough [ 0.716502] SCSI subsystem initialized [ 0.718131] ACPI: bus type USB registered [ 0.720110] usbcore: registered new interface driver usbfs [ 0.722084] usbcore: registered new interface driver hub [ 0.723198] usbcore: registered new device driver usb [ 0.725157] pps_core: LinuxPPS API ver. 1 registered [ 0.727006] pps_core: Software ver. 5.3.6 - Copyright 2005-2007 Rodolfo Giometti [ 0.729053] PTP clock support registered [ 0.732059] EDAC MC: Ver: 3.0.0 [ 0.735127] PCI: Using ACPI for IRQ routing [ 0.736478] NetLabel: Initializing [ 0.737048] NetLabel: domain hash size = 128 [ 0.739010] NetLabel: protocols = UNLABELED CIPSOv4 CALIPSO [ 0.741139] NetLabel: unlabeled traffic allowed by default [ 0.743230] vgaarb: loaded [ 0.745272] hpet0: at MMIO 0xfed00000, IRQs 2, 8, 0 [ 0.747014] hpet0: 3 comparators, 64-bit 100.000000 MHz counter [ 0.757106] clocksource: Switched to clocksource kvm-clock [ 0.864922] VFS: Disk quotas dquot_6.6.0 [ 0.866190] VFS: Dquot-cache hash table entries: 512 (order 0, 4096 bytes) [ 0.868641] *** VALIDATE ramfs *** [ 0.869779] *** VALIDATE hugetlbfs *** [ 0.871676] pnp: PnP ACPI init [ 0.874226] pnp: PnP ACPI: found 6 devices [ 0.893735] clocksource: acpi_pm: mask: 0xffffff max_cycles: 0xffffff, max_idle_ns: 2085701024 ns [ 0.896970] pci_bus 0000:00: resource 4 [io 0x0000-0x0cf7 window] [ 0.899354] pci_bus 0000:00: resource 5 [io 0x0d00-0xffff window] [ 0.901791] pci_bus 0000:00: resource 6 [mem 0x000a0000-0x000bffff window] [ 0.904621] pci_bus 0000:00: resource 7 [mem 0xc0000000-0xfebfffff window] [ 0.907428] pci_bus 0000:00: resource 8 [mem 0xe0000000000-0xe007fffffff window] [ 0.910479] NET: Registered protocol family 2 [ 0.913163] IP idents hash table entries: 131072 (order: 8, 1048576 bytes, vmalloc) [ 0.918390] tcp_listen_portaddr_hash hash table entries: 4096 (order: 5, 163840 bytes, vmalloc) [ 0.922439] TCP established hash table entries: 65536 (order: 7, 524288 bytes, vmalloc) [ 0.928127] TCP bind hash table entries: 65536 (order: 9, 2097152 bytes, vmalloc) [ 0.931863] TCP: Hash tables configured (established 65536 bind 65536) [ 0.935073] MPTCP token hash table entries: 8192 (order: 6, 393216 bytes, vmalloc) [ 0.938459] UDP hash table entries: 4096 (order: 6, 393216 bytes, vmalloc) [ 0.941376] UDP-Lite hash table entries: 4096 (order: 6, 393216 bytes, vmalloc) [ 0.944064] NET: Registered protocol family 1 [ 0.946452] RPC: Registered named UNIX socket transport module. [ 0.949059] RPC: Registered udp transport module. [ 0.950939] RPC: Registered tcp transport module. [ 0.952605] RPC: Registered tcp NFSv4.1 backchannel transport module. [ 0.955286] NET: Registered protocol family 44 [ 0.957196] pci 0000:00:00.0: Limiting direct PCI/PCI transfers [ 0.959274] pci 0000:00:01.0: PIIX3: Enabling Passive Release [ 0.961232] pci 0000:00:01.0: Activating ISA DMA hang workarounds [ 0.962987] PCI: CLS 0 bytes, default 64 [ 0.965061] Unpacking initramfs... [ 2.361861] debug: unmapping init [mem 0xffff92b03cc54000-0xffff92b03ffbffff] [ 2.366336] PCI-DMA: Using software bounce buffering for IO (SWIOTLB) [ 2.368744] software IO TLB: mapped [mem 0x00000000a2000000-0x00000000a6000000] (64MB) [ 2.372124] clocksource: tsc: mask: 0xffffffffffffffff max_cycles: 0x229835b7123, max_idle_ns: 440795242976 ns [ 2.867547] Initialise system trusted keyrings [ 2.868775] Key type blacklist registered [ 2.871458] workingset: timestamp_bits=36 max_order=20 bucket_order=0 [ 2.880646] zbud: loaded [ 2.884817] *** VALIDATE nfs *** [ 2.886275] *** VALIDATE nfs4 *** [ 2.888335] pstore: using deflate compression [ 2.893254] Platform Keyring initialized [ 2.987476] NET: Registered protocol family 38 [ 2.989634] Key type asymmetric registered [ 2.991421] Asymmetric key parser 'x509' registered [ 2.993745] Block layer SCSI generic (bsg) driver version 0.4 loaded (major 247) [ 2.997082] io scheduler mq-deadline registered [ 2.998631] io scheduler kyber registered [ 2.999951] io scheduler bfq registered [ 3.001552] atomic64_test: passed for x86-64 platform with CX8 and with SSE [ 3.004587] shpchp: Standard Hot Plug PCI Controller Driver version: 0.4 [ 3.008419] input: Power Button as /devices/LNXSYSTM:00/LNXPWRBN:00/input/input0 [ 3.011432] ACPI: Power Button [PWRF] [ 3.017096] ACPI: \_SB_.LNKB: Enabled at IRQ 10 [ 3.024247] ACPI: \_SB_.LNKA: Enabled at IRQ 11 [ 3.034538] ACPI: \_SB_.LNKC: Enabled at IRQ 11 [ 3.042177] ACPI: \_SB_.LNKD: Enabled at IRQ 10 [ 3.057802] Serial: 8250/16550 driver, 4 ports, IRQ sharing enabled [ 3.084763] 00:03: ttyS1 at I/O 0x2f8 (irq = 3, base_baud = 115200) is a 16550A [ 3.117572] 00:04: ttyS0 at I/O 0x3f8 (irq = 4, base_baud = 115200) is a 16550A [ 3.122102] Non-volatile memory driver v1.3 [ 3.123764] Linux agpgart interface v0.103 [ 3.155234] virtio_blk virtio1: [vda] 146608 512-byte logical blocks (75.1 MB/71.6 MiB) [ 3.157955] vda: detected capacity change from 0 to 75063296 [ 3.177801] virtio_blk virtio2: [vdb] 2097152 512-byte logical blocks (1.07 GB/1.00 GiB) [ 3.181897] vdb: detected capacity change from 0 to 1073741824 [ 3.194773] virtio_blk virtio3: [vdc] 5120000 512-byte logical blocks (2.62 GB/2.44 GiB) [ 3.198098] vdc: detected capacity change from 0 to 2621440000 [ 3.211416] virtio_blk virtio4: [vdd] 5120000 512-byte logical blocks (2.62 GB/2.44 GiB) [ 3.214832] vdd: detected capacity change from 0 to 2621440000 [ 3.231788] virtio_blk virtio5: [vde] 8388608 512-byte logical blocks (4.29 GB/4.00 GiB) [ 3.235386] vde: detected capacity change from 0 to 4294967296 [ 3.251325] virtio_blk virtio6: [vdf] 8388608 512-byte logical blocks (4.29 GB/4.00 GiB) [ 3.254765] vdf: detected capacity change from 0 to 4294967296 [ 3.262850] libphy: Fixed MDIO Bus: probed [ 3.267923] usbcore: registered new interface driver usbserial_generic [ 3.270566] usbserial: USB Serial support registered for generic [ 3.272989] i8042: PNP: PS/2 Controller [PNP0303:KBD,PNP0f13:MOU] at 0x60,0x64 irq 1,12 [ 3.277399] serio: i8042 KBD port at 0x60,0x64 irq 1 [ 3.279434] serio: i8042 AUX port at 0x60,0x64 irq 12 [ 3.282180] mousedev: PS/2 mouse device common for all mice [ 3.285721] input: AT Translated Set 2 keyboard as /devices/platform/i8042/serio0/input/input1 [ 3.289406] rtc_cmos 00:05: RTC can wake from S4 [ 3.292753] input: VirtualPS/2 VMware VMMouse as /devices/platform/i8042/serio1/input/input4 [ 3.294332] rtc_cmos 00:05: registered as rtc0 [ 3.298143] rtc_cmos 00:05: alarms up to one day, y3k, 242 bytes nvram, hpet irqs [ 3.299161] input: VirtualPS/2 VMware VMMouse as /devices/platform/i8042/serio1/input/input3 [ 3.303287] intel_pstate: CPU model not supported [ 3.311472] hid: raw HID events driver (C) Jiri Kosina [ 3.313622] usbcore: registered new interface driver usbhid [ 3.315635] usbhid: USB HID core driver [ 3.317206] drop_monitor: Initializing network drop monitor service [ 3.319790] Initializing XFRM netlink socket [ 3.321741] NET: Registered protocol family 10 [ 3.324589] Segment Routing with IPv6 [ 3.326360] NET: Registered protocol family 17 [ 3.328729] mpls_gso: MPLS GSO support [ 3.336245] RAS: Correctable Errors collector initialized. [ 3.338669] AVX version of gcm_enc/dec engaged. [ 3.340510] AES CTR mode by8 optimization enabled [ 3.411323] sched_clock: Marking stable (3411303002, 0)->(4333848452, -922545450) [ 3.415122] registered taskstats version 1 [ 3.417797] Loading compiled-in X.509 certificates [ 3.420204] zswap: loaded using pool lzo/zbud [ 3.446541] Key type big_key registered [ 3.460610] Key type encrypted registered [ 3.462387] ima: No TPM chip found, activating TPM-bypass! [ 3.464795] ima: Allocated hash algorithm: sha1 [ 3.466698] ima: No architecture policies found [ 3.468564] evm: Initialising EVM extended attributes: [ 3.470702] evm: security.selinux [ 3.472112] evm: security.ima [ 3.473284] evm: security.capability [ 3.474739] evm: HMAC attrs: 0x1 [ 3.477164] rtc_cmos 00:05: setting system clock to 2026-08-27 00:40:27 UTC (1787791227) [ 3.483515] debug: unmapping init [mem 0xffffffff91a03000-0xffffffff91bfffff] [ 3.487065] debug: unmapping init [mem 0xffffffff90782000-0xffffffff90a58fff] [ 3.497063] Write protecting the kernel read-only data: 28672k [ 3.500284] debug: unmapping init [mem 0xffffffff8ee03000-0xffffffff8effffff] [ 3.502683] debug: unmapping init [mem 0xffffffff8f714000-0xffffffff8f7fffff] [ 3.533132] systemd[1]: systemd 239 (239-82.el8_10.5) running in system mode. (+PAM +AUDIT +SELINUX +IMA -APPARMOR +SMACK +SYSVINIT +UTMP +LIBCRYPTSETUP +GCRYPT +GNUTLS +ACL +XZ +LZ4 +SECCOMP +BLKID +ELFUTILS +KMOD +IDN2 -IDN +PCRE2 default-hierarchy=legacy) [ 3.545452] systemd[1]: Detected virtualization kvm. [ 3.547663] systemd[1]: Detected architecture x86-64. [ 3.549597] systemd[1]: Running in initial RAM disk. Welcome to Rocky Linux 8.10 (Green Obsidian) dracut-049-233.git20240115.el8 (Initramfs)! [ 3.573792] systemd[1]: No hostname configured. [ 3.575426] systemd[1]: Set hostname to . [ 3.577684] random: systemd: uninitialized urandom read (16 bytes read) [ 3.581448] systemd[1]: Initializing machine ID from random generator. [ 3.619301] random: ln: uninitialized urandom read (6 bytes read) [ 3.699512] random: systemd: uninitialized urandom read (16 bytes read) [ 3.702589] systemd[1]: Started Dispatch Password Requests to Console Directory Watch. [ OK ] Started Dispatch Password Requests to Console Directory Watch. [ 3.708381] systemd[1]: Reached target Timers. [ OK ] Reached target Timers. [ 3.712180] systemd[1]: Reached target Initrd Root Device. [ OK ] Reached target Initrd Root Device. [ OK ] Listening on Journal Socket. [ OK ] Listening on udev Kernel Socket. [ OK ] Reached target Local Encrypted Volumes. Starting Create list of required st…ce nodes for the current kernel... [ OK ] Reached target Paths. [ OK ] Listening on Journal Socket (/dev/log). Starting Journal Service... [ OK ] Reached target Swap. [ OK ] Listening on udev Control Socket. [ OK ] Reached target Sockets. [ OK ] Reached target Slices. [ OK ] Started Memstrack Anylazing Service. Starting Setup Virtual Console... Starting Apply Kernel Variables... [ OK ] Reached target Local File Systems. Starting Create Volatile Files and Directories... [ OK ] Started Create list of required sta…vice nodes for the current kernel. [ OK ] Started Setup Virtual Console. [ OK ] Started Apply Kernel Variables. Starting dracut cmdline hook... Starting Create Static Device Nodes in /dev... [ OK ] Started Create Volatile Files and Directories. [ OK ] Started Create Static Device Nodes in /dev. [ OK ] Started Journal Service. [ OK ] Started dracut cmdline hook. Starting dracut pre-udev hook... [ 4.350844] device-mapper: uevent: version 1.0.3 [ 4.353397] device-mapper: ioctl: 4.46.0-ioctl (2022-02-22) initialised: dm-devel@redhat.com [ OK ] Started dracut pre-udev hook. Starting udev Kernel Device Manager... [ OK ] Started udev Kernel Device Manager. Starting dracut pre-trigger hook... [ OK ] Started dracut pre-trigger hook. Starting udev Coldplug all Devices... Mounting Kernel Configuration File System... [ OK ] Mounted Kernel Configuration File System. [ OK ] Started udev Coldplug all Devices. [ OK ] Reached target System Initialization. [ OK ] Reached target Basic System. [ OK ] Started Hardware RNG Entropy Gatherer Daemon. Starting dracut initqueue hook... [ 5.086826] virtio_net virtio0 ens2: renamed from eth0 [ 5.121762] random: fast init done [ 5.196803] scsi host0: ata_piix [ 5.218524] scsi host1: ata_piix [ 5.220463] ata1: PATA max MWDMA2 cmd 0x1f0 ctl 0x3f6 bmdma 0xc320 irq 14 [ 5.225293] ata2: PATA max MWDMA2 cmd 0x170 ctl 0x376 bmdma 0xc328 irq 15 [ 9.004324] dracut-initqueue[584]: RTNETLINK answers: File exists [ 9.980157] random: crng init done [ 9.981869] random: 7 urandom warning(s) missed due to ratelimiting Starting nbd nbd0... [ OK ] Started nbd nbd0. [ OK ] Started dracut initqueue hook. Mounting /sysroot... [ OK ] Reached target Remote File Systems (Pre). [ OK ] Reached target Remote File Systems. [ 10.436243] EXT4-fs (nbd0): mounted filesystem with ordered data mode. Opts: (null) [ OK ] Mounted /sysroot. [ OK ] Reached target Initrd Root File System. Starting Reload Configuration from the Real Root... [ OK ] Started Reload Configuration from the Real Root. [ OK ] Reached target Initrd File Systems. [ OK ] Reached target Initrd Default Target. Starting dracut pre-pivot and cleanup hook... [ OK ] Started dracut pre-pivot and cleanup hook. Starting Cleaning Up and Shutting Down Daemons... [ OK ] Stopped dracut pre-pivot and cleanup hook. [ OK ] Stopped target Initrd Default Target. Stopping Hardware RNG Entropy Gatherer Daemon... [ OK ] Stopped target Remote File Systems. [ OK ] Stopped target Remote File Systems (Pre). [ OK ] Stopped dracut initqueue hook. [ OK ] Stopped target Timers. [ OK ] Stopped target Initrd Root Device. [ OK ] Stopped Hardware RNG Entropy Gatherer Daemon. [ OK ] Stopped target Basic System. [ OK ] Stopped target Sockets. [ OK ] Stopped target Slices. [ OK ] Stopped target Paths. [ OK ] Stopped target System Initialization. [ OK ] Stopped target Local Encrypted Volumes. [ OK ] Stopped Dispatch Password Requests to Console Directory Watch. [ OK ] Stopped Apply Kernel Variables. [ OK ] Stopped Create Volatile Files and Directories. [ OK ] Stopped udev Coldplug all Devices. [ OK ] Stopped dracut pre-trigger hook. Stopping udev Kernel Device Manager... [ OK ] Stopped target Swap. [ OK ] Stopped target Local File Systems. [ OK ] Started Cleaning Up and Shutting Down Daemons. [ OK ] Stopped udev Kernel Device Manager. [ OK ] Stopped Create Static Device Nodes in /dev. [ OK ] Stopped Create list of required sta…vice nodes for the current kernel. [ OK ] Stopped dracut pre-udev hook. [ OK ] Stopped dracut cmdline hook. [ OK ] Closed udev Control Socket. [ OK ] Closed udev Kernel Socket. Starting Cleanup udevd DB... [ OK ] Started Cleanup udevd DB. [ OK ] Reached target Switch Root. Starting Switch Root... [ 11.565120] printk: systemd: 26 output lines suppressed due to ratelimiting [ 11.813962] SELinux: Disabled at runtime. [ 11.872459] systemd[1]: systemd 239 (239-82.el8_10.5) running in system mode. (+PAM +AUDIT +SELINUX +IMA -APPARMOR +SMACK +SYSVINIT +UTMP +LIBCRYPTSETUP +GCRYPT +GNUTLS +ACL +XZ +LZ4 +SECCOMP +BLKID +ELFUTILS +KMOD +IDN2 -IDN +PCRE2 default-hierarchy=legacy) [ 11.882317] systemd[1]: Detected virtualization kvm. [ 11.884364] systemd[1]: Detected architecture x86-64. Welcome to Rocky Linux 8.10 (Green Obsidian)! [ 12.381711] systemd[1]: initrd-switch-root.service: Succeeded. [ 12.385996] systemd[1]: Stopped Switch Root. [ OK ] Stopped Switch Root. [ 12.391321] systemd[1]: systemd-journald.service: Service has no hold-off time (RestartSec=0), scheduling restart. [ 12.396191] systemd[1]: systemd-journald.service: Scheduled restart job, restart counter is at 1. [ 12.400930] systemd[1]: Stopped Journal Service. [ OK ] Stopped Journal Service. [ 12.409308] systemd[1]: Starting Journal Service... Starting Journal Service... [ 12.413788] systemd[1]: Reached target rpc_pipefs.target. [ OK ] Reached target rpc_pipefs.target. [FAILED] Failed to set up automount Arbitrar…rmats File System Automount Point. See 'systemctl status proc-sys-fs-binfmt_misc.automount' for details. [ OK ] Started Dispatch Password Requests to Console Directory Watch. [ OK ] Listening on udev Kernel Socket. [ OK ] Created slice User and Session Slice. [ OK ] Reached target Slices. [ OK ] Created slice system-serial\x2dgetty.slice. Mounting POSIX Message Queue File System... Starting Create list of required st…ce nodes for the current kernel... Activating swap /dev/disk/by-label/SWAP... [ OK ] Created slice system-getty.slice. [ OK ] Stopped target Switch Root. [ 12.495291] Adding 1048572k swap on /dev/vdb. Priority:-2 extents:1 across:1048572k FS [ OK ] Listening on Process Core Dump Socket. [ OK ] Stopped target Initrd File Systems. [ OK ] Started Forward Password Requests to Wall Directory Watch. [ OK ] Reached target Paths. Mounting Huge Pages File System... Starting Remount Root and Kernel File Systems... Mounting Kernel Debug File System... [ OK ] Stopped target Initrd Root File System. [ OK ] Reached target Local Encrypted Volumes. Starting Apply Kernel Variables... [ OK ] Listening on initctl Compatibility Named Pipe. [ OK ] Listening on RPCbind Server Activation Socket. [ OK ] Listening on udev Control Socket. Starting udev Coldplug all Devices... [ OK ] Reached target RPC Port Mapper. [ OK ] Created slice system-sshd\x2dkeygen.slice. [ OK ] Started Journal Service. [ OK ] Mounted POSIX Message Queue File System. [ OK ] Started Create list of required sta…vice nodes for the current kernel. [ OK ] Activated swap /dev/disk/by-label/SWAP. [ OK ] Mounted Huge Pages File System. [FAILED] Failed to start Remount Root and Kernel File Systems. See 'systemctl status systemd-remount-fs.service' for details. [ OK ] Mounted Kernel Debug File System. [ OK ] Started Apply Kernel Variables. Starting Configure read-only root support... [ OK ] Reached target Swap. Starting Create Static Device Nodes in /dev... Starting Flush Journal to Persistent Storage... [ OK ] Started Flush Journal to Persistent Storage. [ OK ] Started Create Static Device Nodes in /dev. [ OK ] Reached target Local File Systems (Pre). Mounting /home/green/git/lustre-release... Mounting /mnt... Starting udev Kernel Device Manager... [ OK ] Mounted /mnt. [ 12.838978] squashfs: version 4.0 (2009/01/31) Phillip Lougher [ OK ] Started udev Coldplug all Devices. [ OK ] Mounted /home/green/git/lustre-release. [ OK ] Started udev Kernel Device Manager. [ 13.148528] piix4_smbus 0000:00:01.3: SMBus Host Controller at 0x700, revision 0 [ 13.193113] input: PC Speaker as /devices/platform/pcspkr/input/input5 [ 13.316613] RAPL PMU: API unit is 2^-32 Joules, 0 fixed counters, 10737418240 ms ovfl timer [ 13.325690] EDAC sbridge: Ver: 1.1.2 [ 14.933838] Key type dns_resolver registered [ 15.236534] NFS: Registering the id_resolver key type [ 15.239089] Key type id_resolver registered [ 15.240856] Key type id_legacy registered [ OK ] Started Configure read-only root support. [ OK ] Reached target Local File Systems. Starting Rebuild Dynamic Linker Cache... Starting Mark the need to relabel after reboot... Starting Load/Save Random Seed... Starting Create Volatile Files and Directories... [ OK ] Started Mark the need to relabel after reboot. [ OK ] Started Load/Save Random Seed. [ OK ] Started Create Volatile Files and Directories. Starting RPC Bind... Starting Update UTMP about System Boot/Shutdown... [ OK ] Started Update UTMP about System Boot/Shutdown. [ OK ] Started RPC Bind. [ OK ] Started Rebuild Dynamic Linker Cache. Starting Update is Completed... [ OK ] Started Update is Completed. [ OK ] Reached target System Initialization. [ OK ] Started dnf makecache --timer. [ OK ] Started Daily Cleanup of Temporary Directories. [ OK ] Listening on D-Bus System Message Bus Socket. [ OK ] Reached target Sockets. [ OK ] Started daily update of the root trust anchor for DNSSEC. [ OK ] Reached target Timers. [ OK ] Reached target Basic System. [ OK ] Started irqbalance daemon. [ OK ] Started D-Bus System Message Bus. Starting Restore /run/initramfs on shutdown... [ OK ] Started Hardware RNG Entropy Gatherer Daemon. Starting Login Service... Starting Network Manager... [ OK ] Reached target sshd-keygen.target. [ OK ] Started Restore /run/initramfs on shutdown. [ OK ] Started Login Service. [ OK ] Started Network Manager. [ OK ] Reached target Network. Starting GSSAPI Proxy Daemon... Starting OpenSSH server daemon... Starting Dynamic System Tuning Daemon... Starting Network Manager Wait Online... Starting Hostname Service... [ OK ] Started GSSAPI Proxy Daemon. [ OK ] Reached target NFS client services. [ OK ] Reached target Remote File Systems (Pre). [ OK ] Reached target Remote File Systems. Starting Permit User Sessions... [ OK ] Started OpenSSH server daemon. [ OK ] Started Permit User Sessions. [ OK ] Started Serial Getty on ttyS0. [ OK ] Started Getty on tty1. [ OK ] Started Serial Getty on ttyS1. [ OK ] Reached target Login Prompts. [ OK ] Started Command Scheduler. [ OK ] Started Hostname Service. Starting Network Manager Script Dispatcher Service... [ OK ] Started Network Manager Script Dispatcher Service. [ OK ] Started Network Manager Wait Online. [ OK ] Reached target Network is Online. Starting Crash recovery kernel arming... Starting System Logging Service... Starting Notify NFS peers of a restart... [ OK ] Started Notify NFS peers of a restart. [ OK ] Started System Logging Service. Starting Authorization Manager... [ OK ] Started Authorization Manager. [ OK ] Started Dynamic System Tuning Daemon. [ OK ] Reached target Multi-User System. [ OK ] Reached target Graphical Interface. Starting Update UTMP about System Runlevel Changes... [ OK ] Started Update UTMP about System Runlevel Changes. Rocky Linux 8.10 (Green Obsidian) Kernel 4.18.0rh8.10-debug on an x86_64 oleg327-server login: [ 40.273503] libcfs: loading out-of-tree module taints kernel. [ 40.297672] Key type ._llcrypt registered [ 40.299628] Key type .llcrypt registered [ 40.357859] Lustre: DEBUG MARKER: oleg327-server.virtnet: executing set_hostid [ 54.479647] Lustre: DEBUG MARKER: oleg327-server.virtnet: executing load_modules_local [ 55.848908] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 55.873247] alg: No test for adler32 (adler32-zlib) [ 57.297886] Lustre: Lustre: Build Version: 2.17.57_50_g504224c [ 58.302520] LNet: Added LNI 192.168.203.127@tcp [8/256/0/180] [ 60.088245] Key type lgssc registered [ 61.986385] Lustre: Echo OBD driver; http://www.lustre.org/ [ 78.655804] ZFS: Loaded module v2.3.2-1, ZFS pool version 5000, ZFS filesystem version 5 [ 100.418010] hrtimer: interrupt took 4000860 ns [ 112.700184] Lustre: DEBUG MARKER: oleg327-server.virtnet: executing load_modules_local [ 124.606115] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 124.656322] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 126.049157] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 126.091171] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 126.202740] Lustre: lustre-MDT0000: new disk, initializing [ 126.283626] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 126.308488] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 130.443231] Lustre: DEBUG MARKER: oleg327-server.virtnet: executing set_default_debug vfstrace rpctrace dlmtrace neterror ha config ioctl super lfsck all [ 141.981041] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 142.065420] Lustre: 6507:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 142.105763] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [ 142.111606] Lustre: Skipped 1 previous similar message [ 142.202706] Lustre: lustre-MDT0001: new disk, initializing [ 142.264276] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 142.293553] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [ 142.309773] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 145.627439] Lustre: DEBUG MARKER: oleg327-server.virtnet: executing set_default_debug vfstrace rpctrace dlmtrace neterror ha config ioctl super lfsck all [ 149.530719] Lustre: Modifying parameter general.debug_raw_pointers=Y in log params [ 157.351773] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 157.569921] Lustre: lustre-OST0000: new disk, initializing [ 157.578521] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 157.585514] Lustre: 8444:0:(osd_compat.c:1353:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 157.657546] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 161.115000] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [ 161.124364] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 161.196754] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 162.324686] Lustre: DEBUG MARKER: oleg327-server.virtnet: executing set_default_debug vfstrace rpctrace dlmtrace neterror ha config ioctl super lfsck all [ 173.516162] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 173.637498] Lustre: lustre-OST0001: new disk, initializing [ 173.643765] Lustre: srv-lustre-OST0001: No data found on store. Initialize space. [ 173.649165] Lustre: 9518:0:(osd_compat.c:1353:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 173.689063] Lustre: lustre-OST0001: Imperative Recovery not enabled, recovery window 60-180 [ 178.585682] Lustre: DEBUG MARKER: oleg327-server.virtnet: executing set_default_debug vfstrace rpctrace dlmtrace neterror ha config ioctl super lfsck all [ 182.303345] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x00000002c0000400-0x0000000300000400]:1:ost [ 182.312980] Lustre: cli-lustre-OST0001-super: Allocated super-sequence [0x00000002c0000400-0x0000000300000400]:1:ost] [ 182.368730] Lustre: lustre-OST0001-osc-MDT0000: update sequence from 0x100010000 to 0x2c0000401 [ 188.717737] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 193.579252] Lustre: Setting parameter general.lod.*.mdt_hash=crush in log params [ 198.596223] Lustre: DEBUG MARKER: oleg327-server.virtnet: executing check_logdir /tmp/testlogs/ [ 203.530353] Lustre: DEBUG MARKER: oleg327-server.virtnet: executing yml_node [ 207.438311] Lustre: DEBUG MARKER: Client: 2.17.57.50 [ 209.728130] Lustre: DEBUG MARKER: MDS: 2.17.57.50 [ 211.932677] Lustre: DEBUG MARKER: OSS: 2.17.57.50 [ 213.546079] Lustre: DEBUG MARKER: -----============= acceptance-small: conf-sanity ============----- Wed Aug 26 20:43:56 EDT 2026 [ 227.891538] Lustre: DEBUG MARKER: excepting tests: 102 106 115 32newtarball [ 229.051332] Lustre: DEBUG MARKER: skipping tests SLOW=no: 45 69 106 111 114 [ 243.683830] Lustre: lustre-MDT0000-lwp-OST0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 243.692909] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 243.698037] Lustre: Skipped 2 previous similar messages [ 243.713620] Lustre: Skipped 3 previous similar messages [ 248.803118] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 248.807970] Lustre: Skipped 3 previous similar messages [ 249.769939] Lustre: server umount lustre-MDT0000 complete [ 252.906540] LustreError: 7455:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1787791476 with bad export cookie 2500111272755672883 [ 252.910382] LustreError: MGC192.168.203.127@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 252.918644] LustreError: 7455:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 1 previous similar message [ 253.921227] Lustre: lustre-MDT0001-lwp-OST0000: Connection to lustre-MDT0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 253.926383] Lustre: lustre-MDT0001: Not available for connect from 0@lo (stopping) [ 253.928340] Lustre: Skipped 2 previous similar messages [ 257.953343] Lustre: lustre-MDT0001: Not available for connect from 0@lo (stopping) [ 257.958815] Lustre: Skipped 2 previous similar messages [ 262.049156] LustreError: 6514:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 262.049335] Lustre: lustre-MDT0001: Not available for connect from 0@lo (stopping) [ 262.069176] LustreError: 6514:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 5 previous similar messages [ 262.090808] Lustre: Skipped 1 previous similar message [ 267.233140] LustreError: 6515:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 267.233709] Lustre: lustre-MDT0001 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 267.250315] LustreError: 6515:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 2 previous similar messages [ 267.380981] Lustre: server umount lustre-MDT0001 complete [ 270.856313] Lustre: server umount lustre-OST0000 complete [ 274.172410] Lustre: server umount lustre-OST0001 complete [ 279.482402] Lustre: DEBUG MARKER: oleg327-server.virtnet: executing set_hostid [ 287.047392] Lustre: DEBUG MARKER: oleg327-server.virtnet: executing load_modules_local [ 315.715188] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 315.968432] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 315.998379] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 316.087130] Lustre: lustre-MDT0000: new disk, initializing [ 316.162961] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 316.175581] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 319.486143] Lustre: DEBUG MARKER: oleg327-server.virtnet: executing set_default_debug -1 all [ 328.945494] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 329.157127] Lustre: 15066:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 329.251929] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [ 329.255498] Lustre: Skipped 1 previous similar message [ 329.329776] Lustre: lustre-MDT0001: new disk, initializing [ 329.413856] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 329.458719] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [ 329.474382] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 334.154907] Lustre: DEBUG MARKER: oleg327-server.virtnet: executing set_default_debug -1 all [ 341.278450] Lustre: DEBUG MARKER: oleg327-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 346.156544] Lustre: DEBUG MARKER: oleg327-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 351.770643] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 352.029272] Lustre: lustre-OST0000: new disk, initializing [ 352.032381] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 352.037568] Lustre: 17037:0:(osd_compat.c:1353:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 352.102425] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 353.843606] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [ 353.855721] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 353.954772] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 356.872346] Lustre: DEBUG MARKER: oleg327-server.virtnet: executing set_default_debug -1 all [ 364.664516] Lustre: DEBUG MARKER: oleg327-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 368.012844] Lustre: DEBUG MARKER: oleg327-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 [ 368.149851] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [ 371.336771] Lustre: DEBUG MARKER: oleg327-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid 50 [ 371.477883] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [ 374.756629] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 374.763443] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 374.763917] LustreError: Skipped 1 previous similar message [ 374.765537] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 374.777603] Lustre: Skipped 1 previous similar message [ 374.779813] Lustre: Skipped 2 previous similar messages [ 379.035282] Lustre: server umount lustre-OST0000 complete [ 384.994425] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 385.008332] Lustre: Skipped 1 previous similar message [ 394.214147] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 394.220082] Lustre: Skipped 6 previous similar messages [ 396.256257] Lustre: lustre-MDT0000 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 396.386122] Lustre: server umount lustre-MDT0000 complete [ 399.128450] LustreError: 15862:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1787791623 with bad export cookie 2500111272755674416 [ 399.133094] LustreError: MGC192.168.203.127@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 399.136823] LustreError: 15862:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 2 previous similar messages [ 399.380524] Lustre: server umount lustre-MDT0001 complete [ 403.045439] Lustre: DEBUG MARKER: == conf-sanity test 76a: set permanent params with lctl across mounts ========================================================== 20:47:06 (1787791626) [ 407.633780] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 407.949516] LustreError: 19302:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 408.002243] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 410.927482] Lustre: DEBUG MARKER: oleg327-server.virtnet: executing set_default_debug -1 all [ 413.156823] LustreError: 19303:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 416.979318] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 420.471414] Lustre: DEBUG MARKER: oleg327-server.virtnet: executing set_default_debug -1 all [ 425.693185] Lustre: DEBUG MARKER: oleg327-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 429.434539] Lustre: DEBUG MARKER: oleg327-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 434.166811] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 438.517699] Lustre: DEBUG MARKER: oleg327-server.virtnet: executing set_default_debug -1 all [ 443.701244] Lustre: DEBUG MARKER: oleg327-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 457.023989] Lustre: Modifying parameter general.osc.*.max_dirty_mb=452 in log params [ 463.331845] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 463.336964] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 463.343941] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 467.843946] Lustre: server umount lustre-MDT0000 complete [ 469.474537] LustreError: 21869:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 469.488213] LustreError: 21869:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 2 previous similar messages [ 470.817739] LustreError: 19283:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1787791694 with bad export cookie 2500111272755675298 [ 470.828520] LustreError: MGC192.168.203.127@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 470.835384] LustreError: 19283:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 471.098293] Lustre: server umount lustre-MDT0001 complete [ 484.838341] Lustre: server umount lustre-OST0000 complete [ 500.948648] Lustre: DEBUG MARKER: oleg327-server.virtnet: executing load_modules_local [ 507.909160] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 508.171798] LustreError: 23699:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 508.215649] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 508.219145] Lustre: Skipped 2 previous similar messages [ 511.341297] Lustre: DEBUG MARKER: oleg327-server.virtnet: executing set_default_debug -1 all [ 517.449229] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 520.611000] Lustre: DEBUG MARKER: oleg327-server.virtnet: executing set_default_debug -1 all [ 522.530722] Lustre: Setting parameter general.debug_raw_pointers=Y in log params [ 527.067615] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 531.483973] Lustre: DEBUG MARKER: oleg327-server.virtnet: executing set_default_debug -1 all [ 538.687060] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 538.848670] Lustre: lustre-OST0001: new disk, initializing [ 538.852468] Lustre: srv-lustre-OST0001: No data found on store. Initialize space. [ 538.857948] Lustre: 26012:0:(osd_compat.c:1353:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 540.105242] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x00000002c0000400-0x0000000300000400]:1:ost [ 540.112377] Lustre: cli-lustre-OST0001-super: Allocated super-sequence [0x00000002c0000400-0x0000000300000400]:1:ost] [ 540.199432] Lustre: lustre-OST0001-osc-MDT0000: update sequence from 0x100010000 to 0x2c0000401 [ 544.589769] Lustre: DEBUG MARKER: oleg327-server.virtnet: executing set_default_debug -1 all [ 552.669582] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 555.185693] Lustre: Setting parameter general.lod.*.mdt_hash=crush in log params [ 563.683357] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 563.691198] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 563.704071] Lustre: Skipped 2 previous similar messages [ 563.711989] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 563.715638] Lustre: Skipped 3 previous similar messages [ 568.140714] Lustre: server umount lustre-MDT0000 complete [ 570.853710] LustreError: 23699:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 570.867097] LustreError: 23699:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 4 previous similar messages [ 570.896783] LustreError: 23680:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1787791794 with bad export cookie 2500111272755676145 [ 570.903191] LustreError: MGC192.168.203.127@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 570.916925] LustreError: 23680:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 571.146221] Lustre: server umount lustre-MDT0001 complete [ 584.216414] Lustre: server umount lustre-OST0000 complete [ 596.195218] Lustre: server umount lustre-OST0001 complete [ 607.777492] Lustre: DEBUG MARKER: oleg327-server.virtnet: executing load_modules_local [ 614.679463] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 614.927633] LustreError: 28982:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 618.250558] Lustre: DEBUG MARKER: oleg327-server.virtnet: executing set_default_debug -1 all [ 624.985654] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 628.379825] Lustre: DEBUG MARKER: oleg327-server.virtnet: executing set_default_debug -1 all [ 630.732673] Lustre: 30129:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify general/debug_raw_pointers=Y (mode = 0) failed: rc = -17 [ 635.470021] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 639.822788] Lustre: DEBUG MARKER: oleg327-server.virtnet: executing set_default_debug -1 all [ 646.020944] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 646.243586] Lustre: lustre-OST0001: Imperative Recovery not enabled, recovery window 60-180 [ 646.251816] Lustre: Skipped 6 previous similar messages [ 650.680761] Lustre: DEBUG MARKER: oleg327-server.virtnet: executing set_default_debug -1 all [ 656.803054] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 660.014424] Lustre: 32008:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify general/lod.*.mdt_hash=crush (mode = 0) failed: rc = -17 [ 666.083297] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 666.086349] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 666.091855] Lustre: Skipped 3 previous similar messages [ 670.713958] Lustre: server umount lustre-MDT0000 complete [ 673.611434] LustreError: 28963:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1787791897 with bad export cookie 2500111272755677433 [ 673.618249] LustreError: MGC192.168.203.127@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 673.632932] LustreError: 28963:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 2 previous similar messages [ 673.909928] Lustre: server umount lustre-MDT0001 complete [ 687.941638] Lustre: server umount lustre-OST0000 complete [ 701.998526] Lustre: server umount lustre-OST0001 complete [ 710.092128] Lustre: DEBUG MARKER: == conf-sanity test 76b: verify params log setup correctly ========================================================== 20:52:13 (1787791933) [ 731.934438] Lustre: DEBUG MARKER: oleg327-server.virtnet: executing load_modules_local [ 739.201543] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 739.532414] LustreError: 34811:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 739.558813] LustreError: 34811:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 11 previous similar messages [ 743.365098] Lustre: DEBUG MARKER: oleg327-server.virtnet: executing set_default_debug -1 all [ 750.743490] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 754.870635] Lustre: DEBUG MARKER: oleg327-server.virtnet: executing set_default_debug -1 all [ 757.548526] Lustre: 35960:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify general/debug_raw_pointers=Y (mode = 0) failed: rc = -17 [ 763.851602] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 768.867706] Lustre: DEBUG MARKER: oleg327-server.virtnet: executing set_default_debug -1 all [ 775.387684] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 780.119667] Lustre: DEBUG MARKER: oleg327-server.virtnet: executing set_default_debug -1 all [ 785.948343] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 788.902472] Lustre: 37838:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify general/lod.*.mdt_hash=crush (mode = 0) failed: rc = -17 [ 797.160703] Lustre: lustre-MDT0000-lwp-OST0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 797.163946] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 797.168655] Lustre: Skipped 6 previous similar messages [ 797.169837] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 797.173281] Lustre: Skipped 12 previous similar messages [ 799.127756] Lustre: server umount lustre-MDT0000 complete [ 802.111080] LustreError: 34791:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1787792026 with bad export cookie 2500111272755678546 [ 802.121606] LustreError: MGC192.168.203.127@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 802.129860] LustreError: 34791:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 802.465759] Lustre: server umount lustre-MDT0001 complete [ 811.945662] Lustre: server umount lustre-OST0000 complete [ 815.915206] Lustre: server umount lustre-OST0001 complete [ 823.657126] Lustre: DEBUG MARKER: == conf-sanity test 76c: verify changelog_mask is applied with lctl set_param -P ========================================================== 20:54:07 (1787792047) [ 837.878340] Lustre: DEBUG MARKER: oleg327-server.virtnet: executing load_modules_local [ 845.703256] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 849.981884] Lustre: DEBUG MARKER: oleg327-server.virtnet: executing set_default_debug -1 all [ 857.219873] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 861.739303] Lustre: DEBUG MARKER: oleg327-server.virtnet: executing set_default_debug -1 all [ 864.263820] Lustre: 41126:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify general/debug_raw_pointers=Y (mode = 0) failed: rc = -17 [ 869.516664] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 871.910382] LustreError: 41481:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 871.927879] LustreError: 41481:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 9 previous similar messages [ 874.846899] Lustre: DEBUG MARKER: oleg327-server.virtnet: executing set_default_debug -1 all [ 886.038496] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 891.978415] Lustre: DEBUG MARKER: oleg327-server.virtnet: executing set_default_debug -1 all [ 898.581693] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 901.525909] Lustre: 43004:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify general/lod.*.mdt_hash=crush (mode = 0) failed: rc = -17 [ 902.306961] Lustre: Setting parameter general.mdd.*.changelog_mask=-CLOSE in log params [ 902.314818] Lustre: Skipped 1 previous similar message [ 906.724365] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 906.732349] Lustre: Skipped 3 previous similar messages [ 910.854757] Lustre: server umount lustre-MDT0000 complete [ 913.573852] LustreError: 41128:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1787792137 with bad export cookie 2500111272755679652 [ 913.580308] LustreError: MGC192.168.203.127@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 913.586253] LustreError: 41128:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 1 previous similar message [ 919.915247] Lustre: server umount lustre-MDT0001 complete [ 926.164417] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 938.470857] Lustre: Evicted from MGS (at 0@lo) after server handle changed from 0x0 to 0x22b22df4d78d560b [ 938.484129] Lustre: MGC192.168.203.127@tcp: Connection restored to 0@lo (at 0@lo) [ 938.806233] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 938.809734] Lustre: Skipped 8 previous similar messages [ 942.413669] Lustre: DEBUG MARKER: oleg327-server.virtnet: executing set_default_debug -1 all [ 948.009069] Lustre: lustre-MDT0000-lwp-OST0000: Connection restored to 0@lo (at 0@lo) [ 952.207756] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 956.684986] Lustre: DEBUG MARKER: oleg327-server.virtnet: executing set_default_debug -1 all [ 957.931703] Lustre: lustre-MDT0001-lwp-OST0000: Connection restored to 0@lo (at 0@lo) [ 957.936367] Lustre: Skipped 1 previous similar message [ 963.115870] Lustre: DEBUG MARKER: oleg327-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 964.596538] Lustre: DEBUG MARKER: mdc.lustre-MDT0000-mdc-*.mds_server_uuid in FULL state after 0 sec [ 971.299519] Lustre: DEBUG MARKER: oleg327-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 973.049654] Lustre: DEBUG MARKER: mdc.lustre-MDT0001-mdc-*.mds_server_uuid in FULL state after 0 sec [ 978.422434] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 982.877181] Lustre: server umount lustre-MDT0000 complete [ 986.922460] LustreError: 39958:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1787792210 with bad export cookie 2500111272755680779 [ 986.927776] LustreError: 39958:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 2 previous similar messages [ 986.930181] LustreError: MGC192.168.203.127@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 987.372449] Lustre: server umount lustre-MDT0001 complete [ 1001.598894] Lustre: server umount lustre-OST0000 complete [ 1004.449416] Lustre: 3643:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1787792212/real 1787792212] req@ffff92b0917dd2c0 x1874635031706368/t0(0) o400->lustre-MDT0001-lwp-OST0001@0@lo:12/10 lens 224/224 e 0 to 1 dl 1787792228 ref 1 fl Rpc:XNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 [ 1006.143314] Lustre: server umount lustre-OST0001 complete [ 1014.585605] Lustre: DEBUG MARKER: == conf-sanity test 76d: verify llite.*.xattr_cache can be set by 'lctl set_param -P' correctly ========================================================== 20:57:17 (1787792237) [ 1030.220379] Lustre: DEBUG MARKER: oleg327-server.virtnet: executing load_modules_local [ 1040.782307] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1044.378967] Lustre: DEBUG MARKER: oleg327-server.virtnet: executing set_default_debug -1 all [ 1051.298191] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1057.026622] Lustre: DEBUG MARKER: oleg327-server.virtnet: executing set_default_debug -1 all [ 1059.672043] Lustre: 48763:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify general/debug_raw_pointers=Y (mode = 0) failed: rc = -17 [ 1065.186030] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 1071.516574] Lustre: DEBUG MARKER: oleg327-server.virtnet: executing set_default_debug -1 all [ 1079.238651] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 1084.088436] Lustre: DEBUG MARKER: oleg327-server.virtnet: executing set_default_debug -1 all [ 1090.134853] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 1093.653131] Lustre: Setting parameter general.llite.*.xattr_cache=0 in log params [ 1102.819747] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 1102.823478] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 1102.836676] Lustre: Skipped 9 previous similar messages [ 1102.842292] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 1102.845653] Lustre: Skipped 16 previous similar messages [ 1108.998310] Lustre: server umount lustre-MDT0000 complete [ 1111.742596] LustreError: 47594:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1787792335 with bad export cookie 2500111272755681577 [ 1111.743387] LustreError: MGC192.168.203.127@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 1111.758374] LustreError: 47594:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 3 previous similar messages [ 1111.959377] Lustre: server umount lustre-MDT0001 complete [ 1124.921446] Lustre: server umount lustre-OST0000 complete [ 1138.501575] Lustre: server umount lustre-OST0001 complete [ 1147.010661] Lustre: DEBUG MARKER: == conf-sanity test 77: comma-separated MGS NIDs and failover node NIDs ========================================================== 20:59:30 (1787792370) [ 1149.109732] Lustre: DEBUG MARKER: SKIP: conf-sanity test_77 mixed loopback and real device not working [ 1150.596670] Lustre: DEBUG MARKER: == conf-sanity test 78: run resize2fs on MDT and OST filesystems ========================================================== 20:59:33 (1787792373) [ 1158.930246] Lustre: DEBUG MARKER: oleg327-server.virtnet: executing set_hostid [ 1165.955078] Lustre: DEBUG MARKER: oleg327-server.virtnet: executing load_modules_local [ 1183.572335] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1183.789121] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 1183.876295] Lustre: lustre-MDT0000: new disk, initializing [ 1183.945401] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 1187.024835] Lustre: DEBUG MARKER: oleg327-server.virtnet: executing set_default_debug -1 all [ 1194.584196] Lustre: DEBUG MARKER: oleg327-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 1199.604042] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 1199.823920] Lustre: 54836:0:(osd_compat.c:1353:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 1201.790218] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:0:ost] [ 1201.820470] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x240000400 [ 1204.511423] Lustre: DEBUG MARKER: oleg327-server.virtnet: executing set_default_debug -1 all [ 1212.057239] Lustre: DEBUG MARKER: oleg327-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 1213.633743] Lustre: DEBUG MARKER: create test files [ 1248.444720] Lustre: server umount lustre-OST0000 complete [ 1251.587753] Lustre: server umount lustre-MDT0000 complete [ 1260.307188] Lustre: DEBUG MARKER: oleg327-server.virtnet: executing unload_modules_local [ 1262.481959] Key type lgssc unregistered [ 1262.734881] LNet: 56486:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 1262.740876] LNetError: 56486:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 1263.786270] LNet: Removed LNI 192.168.203.127@tcp [ 1264.654197] Key type .llcrypt unregistered [ 1264.656962] Key type ._llcrypt unregistered [ 1290.121360] Key type ._llcrypt registered [ 1290.124541] Key type .llcrypt registered [ 1290.278791] Lustre: DEBUG MARKER: oleg327-server.virtnet: executing load_modules_local [ 1292.191326] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 1292.239942] alg: No test for adler32 (adler32-zlib) [ 1293.516157] Lustre: Lustre: Build Version: 2.17.57_50_g504224c [ 1293.858929] LNet: Added LNI 192.168.203.127@tcp [8/256/0/180] [ 1295.552400] Key type lgssc registered [ 1296.756938] Lustre: Echo OBD driver; http://www.lustre.org/ [ 1304.346261] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 1304.362231] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1305.859837] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 1309.823045] Lustre: DEBUG MARKER: oleg327-server.virtnet: executing set_default_debug -1 all [ 1316.270467] Lustre: DEBUG MARKER: oleg327-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 1322.469754] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 1322.750515] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 1328.405840] Lustre: DEBUG MARKER: oleg327-server.virtnet: executing set_default_debug -1 all [ 1330.989702] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x240000400:100 to 0x240000400:129) [ 1331.132173] Lustre: 59197:0:(osd_internal.h:1460:osd_trans_exec_op()) lustre-OST0000: opcode 8: before 509 < left 544, rollback = 9 [ 1331.150798] Lustre: 59197:0:(osd_handler.c:2090:osd_trans_dump_creds()) create: 32/128/6, destroy: 0/0/0 [ 1331.166226] Lustre: 59197:0:(osd_handler.c:2097:osd_trans_dump_creds()) attr_set: 0/0/0, xattr_set: 0/0/0 [ 1331.175773] Lustre: 59197:0:(osd_handler.c:2107:osd_trans_dump_creds()) write: 2/18/1, punch: 0/0/0, quota 1/3/0 [ 1331.195310] Lustre: 59197:0:(osd_handler.c:2114:osd_trans_dump_creds()) insert: 32/544/0, delete: 0/0/0 [ 1331.213291] Lustre: 59197:0:(osd_handler.c:2121:osd_trans_dump_creds()) ref_add: 0/0/0, ref_del: 0/0/0 [ 1335.880860] Lustre: DEBUG MARKER: oleg327-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 1340.814910] Lustre: DEBUG MARKER: check files after expanding the MDT and OST filesystems [ 1345.689407] Lustre: DEBUG MARKER: create more files after expanding the MDT and OST filesystems [ 1351.650115] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 1351.661896] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 1351.681415] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 1356.772261] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 1357.111083] Lustre: server umount lustre-OST0000 complete [ 1360.768271] Lustre: server umount lustre-MDT0000 complete [ 1370.773201] Lustre: DEBUG MARKER: oleg327-server.virtnet: executing unload_modules_local [ 1373.623171] Key type lgssc unregistered [ 1373.961699] LNet: 60815:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 1373.966312] LNetError: 60815:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 1373.992067] LNet: Removed LNI 192.168.203.127@tcp [ 1374.746754] Key type .llcrypt unregistered [ 1374.752146] Key type ._llcrypt unregistered [ 1406.463367] Key type ._llcrypt registered [ 1406.467331] Key type .llcrypt registered [ 1406.612626] Lustre: DEBUG MARKER: oleg327-server.virtnet: executing load_modules_local [ 1408.104793] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 1408.200098] alg: No test for adler32 (adler32-zlib) [ 1409.463281] Lustre: Lustre: Build Version: 2.17.57_50_g504224c [ 1409.940541] LNet: Added LNI 192.168.203.127@tcp [8/256/0/180] [ 1411.752177] Key type lgssc registered [ 1413.808125] Lustre: Echo OBD driver; http://www.lustre.org/ [ 1422.363079] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 1422.377040] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1423.664621] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 1428.552240] Lustre: DEBUG MARKER: oleg327-server.virtnet: executing set_default_debug -1 all [ 1436.361963] Lustre: DEBUG MARKER: oleg327-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 1442.520544] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 1442.875408] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 1448.857131] Lustre: DEBUG MARKER: oleg327-server.virtnet: executing set_default_debug -1 all [ 1449.003952] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x240000400:140 to 0x240000400:161) [ 1449.075767] Lustre: 63652:0:(osd_internal.h:1460:osd_trans_exec_op()) lustre-OST0000: opcode 8: before 509 < left 544, rollback = 9 [ 1449.081936] Lustre: 63652:0:(osd_handler.c:2090:osd_trans_dump_creds()) create: 32/128/6, destroy: 0/0/0 [ 1449.087282] Lustre: 63652:0:(osd_handler.c:2097:osd_trans_dump_creds()) attr_set: 0/0/0, xattr_set: 0/0/0 [ 1449.093145] Lustre: 63652:0:(osd_handler.c:2107:osd_trans_dump_creds()) write: 2/18/1, punch: 0/0/0, quota 1/3/0 [ 1449.103955] Lustre: 63652:0:(osd_handler.c:2114:osd_trans_dump_creds()) insert: 32/544/0, delete: 0/0/0 [ 1449.110012] Lustre: 63652:0:(osd_handler.c:2121:osd_trans_dump_creds()) ref_add: 0/0/0, ref_del: 0/0/0 [ 1456.348995] Lustre: DEBUG MARKER: oleg327-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 1461.507399] Lustre: DEBUG MARKER: check files after shrinking the MDT and OST filesystems [ 1500.129410] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 1500.141314] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 1500.160906] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 1504.876102] Lustre: server umount lustre-OST0000 complete [ 1508.436629] Lustre: server umount lustre-MDT0000 complete [ 1517.340844] Lustre: DEBUG MARKER: oleg327-server.virtnet: executing unload_modules_local [ 1519.697663] Key type lgssc unregistered [ 1519.983448] LNet: 65238:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 1519.993310] LNetError: 65238:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 1520.010232] LNet: Removed LNI 192.168.203.127@tcp [ 1520.707822] Key type .llcrypt unregistered [ 1520.709803] Key type ._llcrypt unregistered [ 1536.243121] Key type ._llcrypt registered [ 1536.244618] Key type .llcrypt registered [ 1536.339217] Lustre: DEBUG MARKER: oleg327-server.virtnet: executing set_hostid [ 1549.733386] Lustre: DEBUG MARKER: oleg327-server.virtnet: executing load_modules_local [ 1550.724380] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 1550.754101] alg: No test for adler32 (adler32-zlib) [ 1551.734617] Lustre: Lustre: Build Version: 2.17.57_50_g504224c [ 1551.915378] LNet: Added LNI 192.168.203.127@tcp [8/256/0/180] [ 1553.552158] Key type lgssc registered [ 1554.640405] Lustre: Echo OBD driver; http://www.lustre.org/ [ 1586.755953] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 1586.787605] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1588.121866] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 1588.155957] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 1588.260534] Lustre: lustre-MDT0000: new disk, initializing [ 1588.356071] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 1588.384903] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 1592.811466] Lustre: DEBUG MARKER: oleg327-server.virtnet: executing set_default_debug -1 all [ 1605.081411] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1605.225948] Lustre: 68392:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 1605.287224] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [ 1605.292402] Lustre: Skipped 1 previous similar message [ 1605.368584] Lustre: lustre-MDT0001: new disk, initializing [ 1605.468775] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 1605.509536] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [ 1605.520542] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 1609.970155] Lustre: DEBUG MARKER: oleg327-server.virtnet: executing set_default_debug -1 all [ 1618.185396] Lustre: DEBUG MARKER: oleg327-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 1623.445393] Lustre: DEBUG MARKER: oleg327-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 1630.185513] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 1630.462379] Lustre: lustre-OST0000: new disk, initializing [ 1630.468839] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 1630.478322] Lustre: 70361:0:(osd_compat.c:1353:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 1630.635620] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 1637.116667] Lustre: DEBUG MARKER: oleg327-server.virtnet: executing set_default_debug -1 all [ 1638.999083] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [ 1639.015899] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 1639.099825] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 1646.902195] Lustre: DEBUG MARKER: oleg327-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 1651.359730] Lustre: DEBUG MARKER: oleg327-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 [ 1651.615966] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [ 1655.656311] Lustre: DEBUG MARKER: oleg327-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid 50 [ 1655.886605] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [ 1659.879134] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 1659.887305] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 1659.887453] Lustre: Skipped 1 previous similar message [ 1659.888539] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 1659.895120] Lustre: Skipped 1 previous similar message [ 1663.627453] Lustre: server umount lustre-OST0000 complete [ 1670.116137] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 1670.121934] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 1670.128129] Lustre: Skipped 1 previous similar message [ 1670.138144] Lustre: Skipped 1 previous similar message [ 1673.844393] Lustre: server umount lustre-MDT0000 complete [ 1677.009939] LustreError: 68385:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1787792901 with bad export cookie 7133353872637851763 [ 1677.010637] LustreError: MGC192.168.203.127@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 1677.020686] LustreError: 68385:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 2 previous similar messages [ 1677.307905] Lustre: server umount lustre-MDT0001 complete [ 1684.535589] Lustre: DEBUG MARKER: == conf-sanity test 79: format MDT/OST without mgs option (should return errors) ========================================================== 21:08:27 (1787792907) [ 1693.398874] Lustre: DEBUG MARKER: oleg327-server.virtnet: executing load_modules_local [ 1729.544755] Lustre: DEBUG MARKER: oleg327-server.virtnet: executing set_hostid [ 1739.857376] Lustre: DEBUG MARKER: oleg327-server.virtnet: executing load_modules_local [ 1777.236741] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1777.407456] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 1777.422573] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 1777.474527] Lustre: lustre-MDT0000: new disk, initializing [ 1777.524512] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 1777.539303] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 1780.791307] Lustre: DEBUG MARKER: oleg327-server.virtnet: executing set_default_debug -1 all [ 1790.275342] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1790.385294] Lustre: 76376:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 1790.442726] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [ 1790.451922] Lustre: Skipped 1 previous similar message [ 1790.535908] Lustre: lustre-MDT0001: new disk, initializing [ 1790.613787] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 1790.645219] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [ 1790.654470] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 1795.262715] Lustre: DEBUG MARKER: oleg327-server.virtnet: executing set_default_debug -1 all [ 1804.305383] Lustre: DEBUG MARKER: oleg327-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 1810.064956] Lustre: DEBUG MARKER: oleg327-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 1818.071097] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 1818.479352] Lustre: lustre-OST0000: new disk, initializing [ 1818.481719] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 1818.511497] Lustre: 78344:0:(osd_compat.c:1353:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 1818.724722] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 1824.379600] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [ 1824.396564] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 1824.522535] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 1827.776387] Lustre: DEBUG MARKER: oleg327-server.virtnet: executing set_default_debug -1 all [ 1838.133315] Lustre: DEBUG MARKER: oleg327-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 1843.786970] Lustre: DEBUG MARKER: oleg327-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 [ 1844.097446] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [ 1849.792804] Lustre: DEBUG MARKER: oleg327-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid 50 [ 1850.278233] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [ 1855.482277] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 1855.493015] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 1855.498422] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 1855.516954] Lustre: Skipped 1 previous similar message [ 1860.590324] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 1860.593529] Lustre: Skipped 2 previous similar messages [ 1865.699165] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 1865.709974] Lustre: Skipped 1 previous similar message [ 1867.233488] Lustre: lustre-OST0000 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 1867.497345] Lustre: server umount lustre-OST0000 complete [ 1872.866105] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 1872.879502] LustreError: Skipped 1 previous similar message [ 1872.893541] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 1875.939960] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 1875.957481] Lustre: Skipped 2 previous similar messages [ 1877.816308] Lustre: server umount lustre-MDT0000 complete [ 1881.062262] LustreError: 76382:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 1881.092196] LustreError: 76382:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 4 previous similar messages [ 1882.214143] LustreError: 76367:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1787793106 with bad export cookie 7133353872637852645 [ 1882.228108] LustreError: MGC192.168.203.127@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 1882.648156] Lustre: server umount lustre-MDT0001 complete [ 1892.347050] Lustre: DEBUG MARKER: == conf-sanity test 80: mgc import reconnect race ======== 21:11:54 (1787793114) [ 1900.577524] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1901.064230] LustreError: 80759:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 1901.129660] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 1906.150720] LustreError: 80760:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 1906.731387] Lustre: DEBUG MARKER: oleg327-server.virtnet: executing set_default_debug -1 all [ 1911.311694] LustreError: 80759:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 1916.386215] LustreError: 80760:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 1917.716171] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1922.871097] Lustre: DEBUG MARKER: oleg327-server.virtnet: executing set_default_debug -1 all [ 1930.752248] Lustre: DEBUG MARKER: oleg327-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 1936.905774] Lustre: DEBUG MARKER: oleg327-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 1944.294402] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 1944.704616] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 1944.711158] Lustre: Skipped 1 previous similar message [ 1951.028457] Lustre: DEBUG MARKER: oleg327-server.virtnet: executing set_default_debug -1 all [ 1958.097266] Lustre: DEBUG MARKER: oleg327-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 1961.841476] Lustre: 83431:0:(genops.c:1774:obd_export_evict_by_uuid()) MGS: evicting 29939380-3a56-4b1d-ae4d-7721962232e2 at adminstrative request [ 1962.477212] LustreError: MGC192.168.203.127@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 1962.499735] Lustre: *** cfs_fail_loc=906, val=2147483648*** [ 1972.769772] Lustre: MGS: Client 29939380-3a56-4b1d-ae4d-7721962232e2 (at 0@lo) reconnecting [ 1972.791827] Lustre: Evicted from MGS (at 0@lo) after server handle changed from 0x62fec38d93bc7f57 to 0x62fec38d93bc81f0 [ 1972.813545] Lustre: MGC192.168.203.127@tcp: Connection restored to 0@lo (at 0@lo) [ 1998.679946] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 1998.846917] Lustre: lustre-OST0001: new disk, initializing [ 1998.852094] Lustre: srv-lustre-OST0001: No data found on store. Initialize space. [ 1998.859189] Lustre: 83824:0:(osd_compat.c:1353:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 2006.024672] Lustre: DEBUG MARKER: oleg327-server.virtnet: executing set_default_debug -1 all [ 2008.785034] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x00000002c0000400-0x0000000300000400]:1:ost [ 2008.800414] Lustre: cli-lustre-OST0001-super: Allocated super-sequence [0x00000002c0000400-0x0000000300000400]:1:ost] [ 2008.961034] Lustre: lustre-OST0001-osc-MDT0000: update sequence from 0x100010000 to 0x2c0000401 [ 2016.248023] Lustre: DEBUG MARKER: oleg327-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0001-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 2019.312407] LustreError: lustre-OST0001-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 2019.312478] Lustre: lustre-OST0001-osc-MDT0000: Connection to lustre-OST0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2019.326440] Lustre: lustre-OST0001: Not available for connect from 0@lo (stopping) [ 2019.344686] Lustre: Skipped 2 previous similar messages [ 2019.362770] Lustre: Skipped 1 previous similar message [ 2024.975071] Lustre: server umount lustre-OST0001 complete [ 2029.034251] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 2029.042270] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2029.554119] LustreError: 82581:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 2029.571364] LustreError: 82581:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 1 previous similar message [ 2043.360145] Lustre: lustre-OST0000 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 2043.630446] Lustre: server umount lustre-OST0000 complete [ 2050.018220] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2050.042687] Lustre: Skipped 2 previous similar messages [ 2055.138693] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 2055.152602] Lustre: Skipped 11 previous similar messages [ 2061.280155] Lustre: lustre-MDT0000 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 2061.409678] Lustre: server umount lustre-MDT0000 complete [ 2065.050435] LustreError: 83823:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1787793289 with bad export cookie 7133353872637854192 [ 2065.051155] LustreError: MGC192.168.203.127@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 2065.063270] LustreError: 83823:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 1 previous similar message [ 2065.391990] Lustre: server umount lustre-MDT0001 complete [ 2075.848624] Lustre: DEBUG MARKER: == conf-sanity test 81: sparse OST indexing ============== 21:14:58 (1787793298) [ 2077.864265] Lustre: DEBUG MARKER: SKIP: conf-sanity test_81 needs >= 3 OSTs [ 2079.869600] Lustre: DEBUG MARKER: == conf-sanity test 82a: specify OSTs for file (succeed) or directory (succeed) ========================================================== 21:15:02 (1787793302) [ 2081.656597] Lustre: DEBUG MARKER: SKIP: conf-sanity test_82a needs >= 3 OSTs [ 2083.861432] Lustre: DEBUG MARKER: == conf-sanity test 82b: specify OSTs for file with --pool and --ost-list options ========================================================== 21:15:06 (1787793306) [ 2086.173200] Lustre: DEBUG MARKER: SKIP: conf-sanity test_82b needs >= 4 OSTs [ 2088.534783] Lustre: DEBUG MARKER: == conf-sanity test 82c: specify sparse OSTs for setstripe (should not crash) ========================================================== 21:15:11 (1787793311) [ 2090.249084] Lustre: DEBUG MARKER: SKIP: conf-sanity test_82c needs >= 4 OSTs [ 2092.239430] Lustre: DEBUG MARKER: == conf-sanity test 83: ENOSPACE on OST doesn't cause message VFS: Busy inodes after unmount ... ========================================================== 21:15:15 (1787793315) [ 2094.317795] Lustre: DEBUG MARKER: mount the OST /dev/mapper/ost1_flakey as a ldiskfs filesystem [ 2101.252727] loop: module loaded [ 2101.274850] loop0: detected capacity change from 0 to 4294967296 [ 2101.343630] LDISKFS-fs (loop0): mounted filesystem with ordered data mode. Opts: (null) [ 2103.121657] Lustre: DEBUG MARKER: run llverfs in partial mode on the OST ldiskfs /mnt/lustre-ost1 [ 2106.550487] Lustre: DEBUG MARKER: oleg327-server.virtnet: executing run_llverfs /mnt/lustre-ost1 -vpl no [ 2110.741338] Lustre: DEBUG MARKER: unmount the OST /dev/mapper/ost1_flakey [ 2138.439636] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 2138.469427] LustreError: 87953:0:(obd_config.c:845:class_setup()) setup lustre-OST0000-osd failed (-28) [ 2138.476421] LustreError: 87953:0:(obd_mount.c:259:lustre_start_simple()) lustre-OST0000-osd setup error -28 [ 2138.483424] LustreError: 87953:0:(tgt_mount.c:2579:server_fill_super()) Unable to start osd on /dev/mapper/ost1_flakey: -28 [ 2138.494756] LustreError: 87953:0:(super25.c:179:lustre_fill_super()) llite: Unable to mount : rc = -28 [ 2157.811466] Lustre: DEBUG MARKER: oleg327-server.virtnet: executing set_hostid [ 2168.888800] Lustre: DEBUG MARKER: oleg327-server.virtnet: executing load_modules_local [ 2206.094150] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 2206.354110] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 2206.412816] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 2206.528440] Lustre: lustre-MDT0000: new disk, initializing [ 2206.629755] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 2206.635370] Lustre: Skipped 1 previous similar message [ 2206.652575] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 2210.777601] Lustre: DEBUG MARKER: oleg327-server.virtnet: executing set_default_debug -1 all [ 2221.138536] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 2221.310846] Lustre: 90817:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 2221.622147] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 2225.505357] Lustre: DEBUG MARKER: oleg327-server.virtnet: executing set_default_debug -1 all [ 2234.829875] Lustre: DEBUG MARKER: oleg327-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 2241.203399] Lustre: DEBUG MARKER: oleg327-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 2247.984853] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 2248.189898] Lustre: 92789:0:(osd_compat.c:1353:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 2254.585448] Lustre: DEBUG MARKER: oleg327-server.virtnet: executing set_default_debug -1 all [ 2255.421982] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 2255.527768] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 2263.857190] Lustre: DEBUG MARKER: oleg327-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 2268.135455] Lustre: DEBUG MARKER: oleg327-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 [ 2268.341643] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [ 2272.169349] Lustre: DEBUG MARKER: oleg327-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid 50 [ 2272.400137] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [ 2276.332111] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 2276.345711] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2276.364306] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 2276.373045] Lustre: Skipped 3 previous similar messages [ 2280.251197] Lustre: server umount lustre-OST0000 complete [ 2289.834078] Lustre: server umount lustre-MDT0000 complete [ 2291.681474] LustreError: 90824:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 2291.695327] LustreError: 90824:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 5 previous similar messages [ 2293.134446] LustreError: 90810:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1787793517 with bad export cookie 7133353872637854815 [ 2293.148048] LustreError: MGC192.168.203.127@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 2293.148497] LustreError: 90810:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 2 previous similar messages [ 2293.469791] Lustre: server umount lustre-MDT0001 complete [ 2301.739800] Lustre: DEBUG MARKER: == conf-sanity test 84: check recovery_hard_time ========= 21:18:44 (1787793524) [ 2308.201650] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 2312.585581] Lustre: DEBUG MARKER: oleg327-server.virtnet: executing set_default_debug -1 all [ 2321.366432] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 2326.241404] Lustre: DEBUG MARKER: oleg327-server.virtnet: executing set_default_debug -1 all [ 2334.404266] Lustre: DEBUG MARKER: oleg327-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 2341.061774] Lustre: DEBUG MARKER: oleg327-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 2347.958164] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 2348.191169] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 2348.202700] Lustre: Skipped 4 previous similar messages [ 2354.787185] Lustre: DEBUG MARKER: oleg327-server.virtnet: executing set_default_debug -1 all [ 2362.825729] Lustre: DEBUG MARKER: oleg327-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 2368.026605] Lustre: DEBUG MARKER: oleg327-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 [ 2368.277396] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [ 2372.918503] Lustre: DEBUG MARKER: oleg327-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid 50 [ 2373.135822] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [ 2380.978142] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 2381.134375] Lustre: lustre-OST0001: new disk, initializing [ 2381.137901] Lustre: Skipped 2 previous similar messages [ 2381.144955] Lustre: srv-lustre-OST0001: No data found on store. Initialize space. [ 2381.151298] Lustre: Skipped 3 previous similar messages [ 2381.154618] Lustre: 98572:0:(osd_compat.c:1353:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 2388.041983] Lustre: DEBUG MARKER: oleg327-server.virtnet: executing set_default_debug -1 all [ 2390.093408] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x00000002c0000400-0x0000000300000400]:1:ost [ 2390.102039] Lustre: Skipped 2 previous similar messages [ 2390.106664] Lustre: cli-lustre-OST0001-super: Allocated super-sequence [0x00000002c0000400-0x0000000300000400]:1:ost] [ 2390.184639] Lustre: lustre-OST0001-osc-MDT0000: update sequence from 0x100010000 to 0x2c0000401 [ 2397.942413] Lustre: DEBUG MARKER: oleg327-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0001-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 2402.879726] Lustre: DEBUG MARKER: oleg327-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0001-osc-MDT0000.ost_server_uuid 50 [ 2403.064749] Lustre: DEBUG MARKER: os[cp].lustre-OST0001-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [ 2407.801112] Lustre: DEBUG MARKER: oleg327-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0001-osc-MDT0001.ost_server_uuid 50 [ 2408.043894] Lustre: DEBUG MARKER: os[cp].lustre-OST0001-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [ 2427.119270] Lustre: DEBUG MARKER: mds1 REPLAY BARRIER on lustre-MDT0000 [ 2458.094526] Lustre: Failing over lustre-MDT0000 [ 2458.684093] Lustre: server umount lustre-MDT0000 complete [ 2460.131978] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 2460.140770] LustreError: Skipped 1 previous similar message [ 2460.148957] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2460.166066] Lustre: Skipped 3 previous similar messages [ 2460.177925] LustreError: 95201:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 2460.199622] LustreError: 95201:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 3 previous similar messages [ 2478.561682] Lustre: 66618:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1787793686/real 1787793686] req@000000005da1a636 x1874636597118976/t0(0) o400->MGC192.168.203.127@tcp@0@lo:26/25 lens 224/224 e 0 to 1 dl 1787793702 ref 1 fl Rpc:XNQr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 [ 2478.587864] LustreError: MGC192.168.203.127@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 2483.255468] LDISKFS-fs (dm-0): 4 truncates cleaned up [ 2483.258549] LDISKFS-fs (dm-0): recovery complete [ 2483.263515] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 2488.807595] LustreError: 66614:0:(client.c:1404:ptlrpc_import_delay_req()) @@@ invalidate in flight req@0000000095d7ad85 x1874636597128192/t0(0) o250->MGC192.168.203.127@tcp@0@lo:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl Rpc:NQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 [ 2489.235771] Lustre: lustre-MDT0000: in recovery but waiting for the first client to connect [ 2492.177140] Lustre: lustre-MDT0000: Will be in recovery for at least 1:00, or until 3 clients reconnect [ 2492.179453] Lustre: 95197:0:(ldlm_lib.c:2110:extend_recovery_timer()) lustre-MDT0000: extended recovery timer reached hard limit: 60, extend: 0 [ 2493.912776] Lustre: DEBUG MARKER: oleg327-server.virtnet: executing set_default_debug -1 all [ 2494.436839] Lustre: 95200:0:(ldlm_lib.c:2110:extend_recovery_timer()) lustre-MDT0000: extended recovery timer reached hard limit: 60, extend: 0 [ 2494.453851] Lustre: 95200:0:(ldlm_lib.c:2110:extend_recovery_timer()) Skipped 1 previous similar message [ 2494.473719] Lustre: lustre-MDT0000-lwp-MDT0001: Connection restored to 0@lo (at 0@lo) [ 2494.548764] LustreError: 101420:0:(ldlm_lib.c:2723:replay_request_or_update()) cfs_fail_timeout id 709 sleeping for 300ms [ 2494.882074] LustreError: 101420:0:(ldlm_lib.c:2723:replay_request_or_update()) cfs_fail_timeout id 709 awake [ 2495.242739] LustreError: 101420:0:(ldlm_lib.c:2723:replay_request_or_update()) cfs_fail_timeout id 709 sleeping for 300ms [ 2495.268957] LustreError: 101420:0:(ldlm_lib.c:2723:replay_request_or_update()) Skipped 1 previous similar message [ 2495.609453] LustreError: 101420:0:(ldlm_lib.c:2723:replay_request_or_update()) cfs_fail_timeout id 709 awake [ 2495.619359] LustreError: 101420:0:(ldlm_lib.c:2723:replay_request_or_update()) Skipped 1 previous similar message [ 2495.648253] Lustre: 101420:0:(ldlm_lib.c:2110:extend_recovery_timer()) lustre-MDT0000: extended recovery timer reached hard limit: 60, extend: 1 [ 2495.667414] Lustre: 101420:0:(ldlm_lib.c:2110:extend_recovery_timer()) Skipped 7 previous similar messages [ 2496.309402] LustreError: 101420:0:(ldlm_lib.c:2723:replay_request_or_update()) cfs_fail_timeout id 709 sleeping for 300ms [ 2496.327878] LustreError: 101420:0:(ldlm_lib.c:2723:replay_request_or_update()) Skipped 2 previous similar messages [ 2496.648421] LustreError: 101420:0:(ldlm_lib.c:2723:replay_request_or_update()) cfs_fail_timeout id 709 awake [ 2496.660223] LustreError: 101420:0:(ldlm_lib.c:2723:replay_request_or_update()) Skipped 2 previous similar messages [ 2497.948979] Lustre: 101420:0:(ldlm_lib.c:2110:extend_recovery_timer()) lustre-MDT0000: extended recovery timer reached hard limit: 60, extend: 1 [ 2497.973619] Lustre: 101420:0:(ldlm_lib.c:2110:extend_recovery_timer()) Skipped 6 previous similar messages [ 2498.619121] LustreError: 101420:0:(ldlm_lib.c:2723:replay_request_or_update()) cfs_fail_timeout id 709 sleeping for 300ms [ 2498.634733] LustreError: 101420:0:(ldlm_lib.c:2723:replay_request_or_update()) Skipped 6 previous similar messages [ 2498.968529] LustreError: 101420:0:(ldlm_lib.c:2723:replay_request_or_update()) cfs_fail_timeout id 709 awake [ 2498.981074] LustreError: 101420:0:(ldlm_lib.c:2723:replay_request_or_update()) Skipped 6 previous similar messages [ 2502.163827] Lustre: 101420:0:(ldlm_lib.c:2110:extend_recovery_timer()) lustre-MDT0000: extended recovery timer reached hard limit: 60, extend: 1 [ 2502.176360] Lustre: 101420:0:(ldlm_lib.c:2110:extend_recovery_timer()) Skipped 12 previous similar messages [ 2502.809033] LustreError: 101420:0:(ldlm_lib.c:2723:replay_request_or_update()) cfs_fail_timeout id 709 sleeping for 300ms [ 2502.824431] LustreError: 101420:0:(ldlm_lib.c:2723:replay_request_or_update()) Skipped 12 previous similar messages [ 2503.168248] LustreError: 101420:0:(ldlm_lib.c:2723:replay_request_or_update()) cfs_fail_timeout id 709 awake [ 2503.190042] LustreError: 101420:0:(ldlm_lib.c:2723:replay_request_or_update()) Skipped 12 previous similar messages [ 2510.193782] Lustre: 101420:0:(ldlm_lib.c:2110:extend_recovery_timer()) lustre-MDT0000: extended recovery timer reached hard limit: 60, extend: 1 [ 2510.206037] Lustre: 101420:0:(ldlm_lib.c:2110:extend_recovery_timer()) Skipped 24 previous similar messages [ 2510.842577] LustreError: 101420:0:(ldlm_lib.c:2723:replay_request_or_update()) cfs_fail_timeout id 709 sleeping for 300ms [ 2510.853478] LustreError: 101420:0:(ldlm_lib.c:2723:replay_request_or_update()) Skipped 24 previous similar messages [ 2511.176124] LustreError: 101420:0:(ldlm_lib.c:2723:replay_request_or_update()) cfs_fail_timeout id 709 awake [ 2511.179885] LustreError: 101420:0:(ldlm_lib.c:2723:replay_request_or_update()) Skipped 24 previous similar messages [ 2526.378336] Lustre: 101420:0:(ldlm_lib.c:2110:extend_recovery_timer()) lustre-MDT0000: extended recovery timer reached hard limit: 60, extend: 1 [ 2526.402414] Lustre: 101420:0:(ldlm_lib.c:2110:extend_recovery_timer()) Skipped 50 previous similar messages [ 2527.057064] LustreError: 101420:0:(ldlm_lib.c:2723:replay_request_or_update()) cfs_fail_timeout id 709 sleeping for 300ms [ 2527.076363] LustreError: 101420:0:(ldlm_lib.c:2723:replay_request_or_update()) Skipped 50 previous similar messages [ 2527.401458] LustreError: 101420:0:(ldlm_lib.c:2723:replay_request_or_update()) cfs_fail_timeout id 709 awake [ 2527.420790] LustreError: 101420:0:(ldlm_lib.c:2723:replay_request_or_update()) Skipped 50 previous similar messages [ 2552.176672] Lustre: 101420:0:(ldlm_lib.c:2419:target_recovery_overseer()) lustre-MDT0000 recovery is aborted by hard timeout [ 2552.198052] Lustre: 101420:0:(ldlm_lib.c:2429:target_recovery_overseer()) recovery is aborted, evict exports in recovery [ 2552.211938] Lustre: 101420:0:(genops.c:1601:class_disconnect_stale_exports()) lustre-MDT0000: disconnect stale client 96860254-2acb-4ce2-abb2-20dda9314704@192.168.203.27@tcp [ 2552.230814] Lustre: lustre-MDT0000: disconnecting 1 stale clients [ 2552.253557] Lustre: 101420:0:(ldlm_lib.c:1939:abort_req_replay_queue()) @@@ aborted: req@00000000780efe7e x1874636584326272/t0(8589934777) o101->96860254-2acb-4ce2-abb2-20dda9314704@192.168.203.27@tcp:596/0 lens 592/0 e 0 to 0 dl 1787793786 ref 1 fl Complete:/604/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:0 [ 2552.313256] Lustre: lustre-MDT0000-osd: cancel update llog [0x200000400:0x1:0x0] [ 2552.323689] Lustre: lustre-MDT0000: Denying connection for new client 96860254-2acb-4ce2-abb2-20dda9314704 (at 192.168.203.27@tcp), waiting for 3 known clients (2 recovered, 0 in progress, and 1 evicted) to recover in 0:00 [ 2552.334824] Lustre: lustre-MDT0001-osp-MDT0000: cancel update llog [0x240000401:0x1:0x0] [ 2552.375948] Lustre: lustre-MDT0000: Recovery over after 1:00, of 3 clients 2 recovered and 1 was evicted. [ 2552.379822] Lustre: lustre-MDT0000-osp-MDT0001: Connection restored to 0@lo (at 0@lo) [ 2552.397377] Lustre: Skipped 2 previous similar messages [ 2552.417541] Lustre: lustre-OST0001: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x2c0000401:47 to 0x2c0000401:545) [ 2552.422411] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:79 to 0x280000401:577) [ 2571.237903] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 2571.242730] Lustre: Skipped 3 previous similar messages [ 2583.013714] Lustre: lustre-OST0000 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 2583.183375] Lustre: server umount lustre-OST0000 complete [ 2586.604427] LustreError: 101787:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-OST0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 2586.620132] LustreError: 101787:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 41 previous similar messages [ 2589.153452] LustreError: lustre-OST0001-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 2589.163720] Lustre: lustre-OST0001-osc-MDT0000: Connection to lustre-OST0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2589.176100] Lustre: Skipped 5 previous similar messages [ 2593.394751] Lustre: server umount lustre-OST0001 complete [ 2603.448925] Lustre: server umount lustre-MDT0000 complete [ 2607.312399] LustreError: 95181:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1787793831 with bad export cookie 7133353872637906034 [ 2607.319108] LustreError: MGC192.168.203.127@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 2607.323866] LustreError: 95181:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 2607.799711] Lustre: server umount lustre-MDT0001 complete [ 2617.173730] Lustre: DEBUG MARKER: == conf-sanity test 85: osd_ost init: fail ea_fid_set ==== 21:24:00 (1787793840) [ 2625.573287] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 2625.586354] Lustre: *** cfs_fail_loc=197, val=0*** [ 2641.248957] LustreError: Failed to get MGS log lustre-sptlrpc and no local copy. [ 2646.368460] LustreError: 103406:0:(mgc_request.c:233:do_config_log_add()) MGC192.168.203.127@tcp: failed processing log, type 4: rc = -110 [ 2651.579731] Lustre: *** cfs_fail_loc=197, val=0*** [ 2651.586743] Lustre: Skipped 65 previous similar messages [ 2672.096312] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 2672.103063] Lustre: Skipped 2 previous similar messages [ 2678.504924] Lustre: DEBUG MARKER: oleg327-server.virtnet: executing set_default_debug -1 all [ 2687.147574] Lustre: DEBUG MARKER: oleg327-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 2689.490800] Lustre: server umount lustre-OST0000 complete [ 2699.259433] Lustre: DEBUG MARKER: == conf-sanity test 86: Replacing mkfs.lustre -G option == 21:25:22 (1787793922) [ 2721.156937] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 2721.646524] LustreError: 105027:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 2721.666101] LustreError: 105027:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 4 previous similar messages [ 2726.085132] Lustre: DEBUG MARKER: oleg327-server.virtnet: executing set_default_debug -1 all [ 2736.524516] Lustre: DEBUG MARKER: oleg327-client.virtnet: executing wait_import_state_mount (FULL|IDLE) mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 2741.233376] LustreError: 105058:0:(osp_object.c:618:osp_attr_get()) lustre-MDT0001-osp-MDT0000: osp_attr_get update error [0x200000009:0x1:0x0]: rc = -5 [ 2741.247844] LustreError: 105058:0:(lod_sub_object.c:917:lod_sub_prep_llog()) lustre-MDT0000-mdtlov: can't get id from catalogs: rc = -5 [ 2741.259967] LustreError: 105058:0:(lod_dev.c:511:lod_sub_recovery_thread()) lustre-MDT0001-osp-MDT0000: get update log duration 20, retries 0, failed: rc = -5 [ 2741.532872] Lustre: server umount lustre-MDT0000 complete [ 2777.627759] Lustre: DEBUG MARKER: oleg327-server.virtnet: executing set_hostid [ 2794.870954] Lustre: DEBUG MARKER: oleg327-server.virtnet: executing load_modules_local [ 2832.929247] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 2833.123254] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 2833.144071] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 2833.190980] Lustre: lustre-MDT0000: new disk, initializing [ 2833.249604] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 2837.270752] Lustre: DEBUG MARKER: oleg327-server.virtnet: executing set_default_debug -1 all [ 2848.697963] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 2848.875166] Lustre: 109264:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 2849.377862] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 2853.839350] Lustre: DEBUG MARKER: oleg327-server.virtnet: executing set_default_debug -1 all [ 2864.356364] Lustre: DEBUG MARKER: oleg327-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 2870.938496] Lustre: DEBUG MARKER: oleg327-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 2878.235543] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 2878.535876] Lustre: 111230:0:(osd_compat.c:1353:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 2885.340133] Lustre: DEBUG MARKER: oleg327-server.virtnet: executing set_default_debug -1 all [ 2888.438861] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 2895.972482] Lustre: DEBUG MARKER: oleg327-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 2900.883926] Lustre: DEBUG MARKER: oleg327-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 [ 2901.089736] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [ 2905.448733] Lustre: DEBUG MARKER: oleg327-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid 50 [ 2905.671424] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [ 2909.153922] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 2909.161119] LustreError: Skipped 2 previous similar messages [ 2909.164498] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2909.173878] Lustre: Skipped 3 previous similar messages [ 2909.182232] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 2909.186095] Lustre: Skipped 12 previous similar messages [ 2913.726197] Lustre: server umount lustre-OST0000 complete [ 2923.504624] Lustre: server umount lustre-MDT0000 complete [ 2927.088488] LustreError: 109255:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1787794151 with bad export cookie 7133353872637916688 [ 2927.098325] LustreError: MGC192.168.203.127@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 2927.121200] LustreError: 109255:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 2 previous similar messages [ 2927.406637] Lustre: server umount lustre-MDT0001 complete [ 2936.911022] Lustre: DEBUG MARKER: == conf-sanity test 87: check if MDT inode can hold EAs with N stripes properly ========================================================== 21:29:19 (1787794159) [ 2946.882810] Lustre: DEBUG MARKER: oleg327-server.virtnet: executing unload_modules_local [ 2949.314303] Key type lgssc unregistered [ 2949.574734] LNet: 113720:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 2949.581452] LNetError: 113720:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 2949.599242] LNet: Removed LNI 192.168.203.127@tcp [ 2950.512094] Key type .llcrypt unregistered [ 2950.513903] Key type ._llcrypt unregistered [ 2969.531583] Key type ._llcrypt registered [ 2969.538074] Key type .llcrypt registered [ 2969.639050] Lustre: DEBUG MARKER: oleg327-server.virtnet: executing set_hostid [ 2984.949937] Lustre: DEBUG MARKER: oleg327-server.virtnet: executing load_modules_local [ 2985.952317] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 2986.121583] alg: No test for adler32 (adler32-zlib) [ 2987.362473] Lustre: Lustre: Build Version: 2.17.57_50_g504224c [ 2987.669532] LNet: Added LNI 192.168.203.127@tcp [8/256/0/180] [ 2989.376258] Key type lgssc registered [ 2990.644235] Lustre: Echo OBD driver; http://www.lustre.org/ [ 3036.745938] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 3036.764042] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 3038.030938] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 3038.060841] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 3038.129784] Lustre: lustre-MDT0000: new disk, initializing [ 3038.231484] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 3038.255747] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 3043.109804] Lustre: DEBUG MARKER: oleg327-server.virtnet: executing set_default_debug -1 all [ 3053.968640] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 3054.191939] Lustre: lustre-OST0000: new disk, initializing [ 3054.195561] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 3054.199434] Lustre: Skipped 1 previous similar message [ 3054.203991] Lustre: 117893:0:(osd_compat.c:1353:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 3054.264597] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 3061.502422] Lustre: DEBUG MARKER: oleg327-server.virtnet: executing set_default_debug -1 all [ 3062.328919] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:0:ost [ 3062.339970] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:0:ost] [ 3062.359161] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x240000400 [ 3071.901702] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 3072.113579] Lustre: lustre-OST0001: new disk, initializing [ 3072.122098] Lustre: srv-lustre-OST0001: No data found on store. Initialize space. [ 3072.133191] Lustre: 118759:0:(osd_compat.c:1353:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 3072.216052] Lustre: lustre-OST0001: Imperative Recovery not enabled, recovery window 60-180 [ 3078.848067] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:1:ost [ 3078.857556] Lustre: cli-lustre-OST0001-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:1:ost] [ 3078.871919] Lustre: DEBUG MARKER: oleg327-server.virtnet: executing set_default_debug -1 all [ 3078.922235] Lustre: lustre-OST0001-osc-MDT0000: update sequence from 0x100010000 to 0x280000400 [ 3089.379250] Lustre: lustre-MDT0000-lwp-OST0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 3089.380460] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 3089.391174] Lustre: Skipped 1 previous similar message [ 3089.415766] Lustre: Skipped 1 previous similar message [ 3094.209814] Lustre: server umount lustre-MDT0000 complete [ 3100.214592] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: (null) [ 3104.440539] LustreError: 118756:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1787794328 with bad export cookie 13764886317177864163 [ 3104.445422] LustreError: MGC192.168.203.127@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 3104.450590] LustreError: 118756:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 1 previous similar message [ 3104.511652] Lustre: server umount lustre-OST0000 complete [ 3108.122278] Lustre: server umount lustre-OST0001 complete [ 3117.313637] Lustre: DEBUG MARKER: == conf-sanity test 88a: tunefs.lustre persists a --mountfsoptions-only change ========================================================== 21:32:20 (1787794340) [ 3139.831672] Lustre: DEBUG MARKER: == conf-sanity test 88b: check the default mount options can be overridden ========================================================== 21:32:42 (1787794362) [ 3150.260578] Lustre: DEBUG MARKER: oleg327-server.virtnet: executing load_modules_local [ 3174.374805] Lustre: DEBUG MARKER: == conf-sanity test 89: check tunefs --param and --erase-param{s} options ========================================================== 21:33:17 (1787794397) [ 3217.445623] Lustre: DEBUG MARKER: oleg327-server.virtnet: executing set_hostid [ 3229.968737] Lustre: DEBUG MARKER: oleg327-server.virtnet: executing load_modules_local [ 3271.869954] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 3272.174497] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 3272.244155] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 3272.361478] Lustre: lustre-MDT0000: new disk, initializing [ 3272.497492] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 3272.519421] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 3277.074427] Lustre: DEBUG MARKER: oleg327-server.virtnet: executing set_default_debug -1 all [ 3287.366476] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 3287.513880] Lustre: 126511:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 3287.616134] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [ 3287.622326] Lustre: Skipped 1 previous similar message [ 3287.720782] Lustre: lustre-MDT0001: new disk, initializing [ 3287.800910] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 3287.845677] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [ 3287.854822] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 3292.482124] Lustre: DEBUG MARKER: oleg327-server.virtnet: executing set_default_debug -1 all [ 3302.244434] Lustre: DEBUG MARKER: oleg327-client.virtnet: executing wait_import_state_mount FULL failover.node=192.0.2.254@tcp0,mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 3307.997628] Lustre: DEBUG MARKER: oleg327-client.virtnet: executing wait_import_state_mount FULL failover.node=192.0.2.254@tcp0,mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 3315.094409] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 3315.406692] Lustre: lustre-OST0000: new disk, initializing [ 3315.413512] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 3315.435800] Lustre: 128479:0:(osd_compat.c:1353:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 3315.566450] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 3321.923424] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [ 3321.941596] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 3322.082876] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 3322.197954] Lustre: DEBUG MARKER: oleg327-server.virtnet: executing set_default_debug -1 all [ 3332.887928] Lustre: DEBUG MARKER: oleg327-client.virtnet: executing wait_import_state_mount (FULL|IDLE) failover.node=192.0.2.254@tcp0,osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 3338.317591] Lustre: DEBUG MARKER: oleg327-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 [ 3338.540317] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [ 3343.672472] Lustre: DEBUG MARKER: oleg327-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid 50 [ 3343.908739] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [ 3347.941270] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 3347.952138] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 3347.958940] LustreError: Skipped 1 previous similar message [ 3347.960865] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 3347.991306] Lustre: Skipped 1 previous similar message [ 3353.064124] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 3353.072840] Lustre: Skipped 2 previous similar messages [ 3358.217678] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 3358.232365] Lustre: Skipped 1 previous similar message [ 3360.224136] Lustre: lustre-OST0000 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 3360.656601] Lustre: server umount lustre-OST0000 complete [ 3368.418307] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 3368.419600] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 3368.435822] Lustre: Skipped 1 previous similar message [ 3368.447964] Lustre: Skipped 1 previous similar message [ 3371.797854] Lustre: server umount lustre-MDT0000 complete [ 3376.058503] LustreError: 126504:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1787794600 with bad export cookie 13764886317177865381 [ 3376.059424] LustreError: MGC192.168.203.127@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 3376.083431] LustreError: 126504:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 1 previous similar message [ 3376.573924] Lustre: server umount lustre-MDT0001 complete [ 3385.926635] Lustre: DEBUG MARKER: == conf-sanity test 90a: check max_mod_rpcs_in_flight is enforced ========================================================== 21:36:48 (1787794608) [ 3392.751509] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 3393.144060] LustreError: 130896:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 3393.161283] LustreError: 130896:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 3 previous similar messages [ 3393.197942] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 3397.846853] Lustre: DEBUG MARKER: oleg327-server.virtnet: executing set_default_debug -1 all [ 3398.628529] LustreError: 130897:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 3403.745932] LustreError: 130896:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 3406.807913] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 3412.168354] Lustre: DEBUG MARKER: oleg327-server.virtnet: executing set_default_debug -1 all [ 3419.725875] Lustre: DEBUG MARKER: oleg327-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 3425.552710] Lustre: DEBUG MARKER: oleg327-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 3432.960846] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 3433.350886] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 3433.361907] Lustre: Skipped 1 previous similar message [ 3440.541679] Lustre: DEBUG MARKER: oleg327-server.virtnet: executing set_default_debug -1 all [ 3448.078351] Lustre: DEBUG MARKER: oleg327-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 3455.836773] Lustre: *** cfs_fail_loc=159, val=0*** [ 3472.208837] Lustre: lustre-MDT0000: Client 7caf9d73-e657-4208-835b-1e277670ff6e (at 192.168.203.27@tcp) reconnecting [ 3474.046173] Lustre: *** cfs_fail_loc=159, val=0*** [ 3474.054230] Lustre: Skipped 5 previous similar messages [ 3489.602801] Lustre: lustre-MDT0000: Client 7caf9d73-e657-4208-835b-1e277670ff6e (at 192.168.203.27@tcp) reconnecting [ 3494.887887] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 3494.901181] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 3494.920659] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 3496.939504] Lustre: server umount lustre-OST0000 complete [ 3504.097621] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 3504.135605] Lustre: Skipped 1 previous similar message [ 3507.018361] Lustre: server umount lustre-MDT0000 complete [ 3509.224471] LustreError: 130897:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 3509.255530] LustreError: 130897:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 1 previous similar message [ 3510.777317] LustreError: 130877:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1787794734 with bad export cookie 13764886317177866270 [ 3510.778917] LustreError: MGC192.168.203.127@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 3510.797044] LustreError: 130877:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 3521.282359] Lustre: server umount lustre-MDT0001 complete [ 3535.768701] Lustre: DEBUG MARKER: oleg327-server.virtnet: executing unload_modules_local [ 3539.407315] Key type lgssc unregistered [ 3539.786623] LNet: 134675:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 3539.799500] LNetError: 134675:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 3539.819750] LNet: Removed LNI 192.168.203.127@tcp [ 3541.125367] Key type .llcrypt unregistered [ 3541.129125] Key type ._llcrypt unregistered [ 3566.750830] Key type ._llcrypt registered [ 3566.752766] Key type .llcrypt registered [ 3566.894831] Lustre: DEBUG MARKER: oleg327-server.virtnet: executing load_modules_local [ 3568.389433] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 3568.692058] alg: No test for adler32 (adler32-zlib) [ 3569.990499] Lustre: Lustre: Build Version: 2.17.57_50_g504224c [ 3570.305994] LNet: Added LNI 192.168.203.127@tcp [8/256/0/180] [ 3572.050825] Key type lgssc registered [ 3573.732483] Lustre: Echo OBD driver; http://www.lustre.org/ [ 3582.195579] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 3582.217996] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 3583.816067] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 3588.395674] Lustre: DEBUG MARKER: oleg327-server.virtnet: executing set_default_debug -1 all [ 3593.186885] LustreError: 136091:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 3593.219670] LustreError: 136091:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 2 previous similar messages [ 3596.886706] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 3597.285166] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 3601.497702] Lustre: DEBUG MARKER: oleg327-server.virtnet: executing set_default_debug -1 all [ 3609.417966] Lustre: DEBUG MARKER: oleg327-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 3615.285160] Lustre: DEBUG MARKER: oleg327-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 3621.891473] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 3622.345011] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 3628.614227] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:42 to 0x280000401:65) [ 3629.667574] Lustre: DEBUG MARKER: oleg327-server.virtnet: executing set_default_debug -1 all [ 3638.905968] Lustre: DEBUG MARKER: oleg327-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 3649.668041] Lustre: *** cfs_fail_loc=159, val=0*** [ 3666.234033] Lustre: lustre-MDT0000: Client f19e071c-5e0e-4a8f-9be2-6dd80c0ce75f (at 192.168.203.27@tcp) reconnecting [ 3669.315578] Lustre: *** cfs_fail_loc=159, val=0*** [ 3684.662432] Lustre: lustre-MDT0001: Client f19e071c-5e0e-4a8f-9be2-6dd80c0ce75f (at 192.168.203.27@tcp) reconnecting [ 3686.490482] Lustre: *** cfs_fail_loc=159, val=0*** [ 3686.498943] Lustre: Skipped 4 previous similar messages [ 3702.093406] Lustre: lustre-MDT0001: Client f19e071c-5e0e-4a8f-9be2-6dd80c0ce75f (at 192.168.203.27@tcp) reconnecting [ 3707.347057] Lustre: 139130:0:(mdt_handler.c:64:mdt_max_mod_rpcs_per_client_set()) max_mod_rpcs_per_client is deprecated, set mdt.*.max_mod_rpcs_in_flight parameter instead [ 3710.404864] Lustre: *** cfs_fail_loc=159, val=0*** [ 3710.416712] Lustre: Skipped 3 previous similar messages [ 3725.618445] Lustre: lustre-MDT0000: Client 393dd882-f629-48b5-aa6f-3713b981bacd (at 192.168.203.27@tcp) reconnecting [ 3727.369678] Lustre: *** cfs_fail_loc=159, val=0*** [ 3727.375888] Lustre: Skipped 15 previous similar messages [ 3743.055837] Lustre: lustre-MDT0000: Client 393dd882-f629-48b5-aa6f-3713b981bacd (at 192.168.203.27@tcp) reconnecting [ 3744.799965] Lustre: 139423:0:(mdt_handler.c:64:mdt_max_mod_rpcs_per_client_set()) max_mod_rpcs_per_client is deprecated, set mdt.*.max_mod_rpcs_in_flight parameter instead [ 3750.887321] LustreError: lustre-OST0000-osc-MDT0001: operation ost_destroy to node 0@lo failed: rc = -107 [ 3750.896288] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 3750.906663] LustreError: Skipped 1 previous similar message [ 3750.949049] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 3751.393607] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 3751.405494] LustreError: Skipped 2 previous similar messages [ 3751.408113] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 3756.521713] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 3756.523831] Lustre: Skipped 2 previous similar messages [ 3756.971879] Lustre: server umount lustre-OST0000 complete [ 3761.121993] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 3761.136570] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 3761.154263] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 3766.754112] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 3766.759213] Lustre: Skipped 2 previous similar messages [ 3767.124782] Lustre: server umount lustre-MDT0000 complete [ 3770.991916] LustreError: 136826:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1787794995 with bad export cookie 259666705083128379 [ 3771.008876] LustreError: MGC192.168.203.127@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 3771.015650] LustreError: 136826:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 3 previous similar messages [ 3771.609332] Lustre: server umount lustre-MDT0001 complete [ 3784.571183] Lustre: DEBUG MARKER: oleg327-server.virtnet: executing unload_modules_local [ 3787.874684] Key type lgssc unregistered [ 3788.345839] LNet: 140482:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 3788.355719] LNetError: 140482:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 3788.384691] LNet: Removed LNI 192.168.203.127@tcp [ 3789.306262] Key type .llcrypt unregistered [ 3789.311733] Key type ._llcrypt unregistered [ 3817.878224] Key type ._llcrypt registered [ 3817.880525] Key type .llcrypt registered [ 3818.031728] Lustre: DEBUG MARKER: oleg327-server.virtnet: executing load_modules_local [ 3819.558599] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 3819.603707] alg: No test for adler32 (adler32-zlib) [ 3820.765937] Lustre: Lustre: Build Version: 2.17.57_50_g504224c [ 3821.128382] LNet: Added LNI 192.168.203.127@tcp [8/256/0/180] [ 3822.912153] Key type lgssc registered [ 3824.626268] Lustre: Echo OBD driver; http://www.lustre.org/ [ 3834.826798] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 3834.875069] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 3836.554041] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 3841.767815] Lustre: DEBUG MARKER: oleg327-server.virtnet: executing set_default_debug -1 all [ 3846.121197] LustreError: 141902:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 3846.142930] LustreError: 141902:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 2 previous similar messages [ 3850.698982] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 3851.104604] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 3856.059893] Lustre: DEBUG MARKER: oleg327-server.virtnet: executing set_default_debug -1 all [ 3864.038097] Lustre: DEBUG MARKER: oleg327-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 3870.203806] Lustre: DEBUG MARKER: oleg327-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 3877.320197] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 3877.652568] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 3881.809472] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:85 to 0x280000401:129) [ 3884.118700] Lustre: DEBUG MARKER: oleg327-server.virtnet: executing set_default_debug -1 all [ 3891.904728] Lustre: DEBUG MARKER: oleg327-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 3896.154180] Lustre: lustre-OST0000: new connection from lustre-MDT0001-mdtlov (cleaning up unused objects from 0x280000400:72 to 0x280000400:97) [ 3901.790693] Lustre: 144486:0:(mdt_handler.c:64:mdt_max_mod_rpcs_per_client_set()) max_mod_rpcs_per_client is deprecated, set mdt.*.max_mod_rpcs_in_flight parameter instead [ 3905.807245] Lustre: 144534:0:(mdt_handler.c:64:mdt_max_mod_rpcs_per_client_set()) max_mod_rpcs_per_client is deprecated, set mdt.*.max_mod_rpcs_in_flight parameter instead [ 3907.785607] Lustre: Modifying parameter lustre.mdc.lustre-MDT0000-mdc-*.max_rpcs_in_flight=8 in log params [ 3908.918818] Lustre: Setting parameter lustre.mdc.lustre-MDT0000-mdc-*.max_mod_rpcs_in_flight=8 in log params [ 3912.940662] Lustre: 144682:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre/mdc.lustre-MDT0000-mdc-*.max_rpcs_in_flight=8 (mode = 0) failed: rc = -17 [ 3914.153511] Lustre: Modifying parameter lustre.mdc.lustre-MDT0000-mdc-*.max_mod_rpcs_in_flight=7 in log params [ 3915.355149] Lustre: 144781:0:(mdt_handler.c:64:mdt_max_mod_rpcs_per_client_set()) max_mod_rpcs_per_client is deprecated, set mdt.*.max_mod_rpcs_in_flight parameter instead [ 3921.889980] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 3921.892792] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 3921.894334] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 3921.928395] Lustre: Skipped 1 previous similar message [ 3926.001317] Lustre: server umount lustre-OST0000 complete [ 3932.131075] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 3932.146730] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 3932.159655] Lustre: Skipped 1 previous similar message [ 3932.168972] Lustre: Skipped 1 previous similar message [ 3937.251894] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 3937.271177] Lustre: Skipped 1 previous similar message [ 3938.715306] Lustre: server umount lustre-MDT0000 complete [ 3942.380606] LustreError: 141898:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 3942.414380] LustreError: 141898:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 1 previous similar message [ 3943.399878] LustreError: 141901:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 3943.437333] LustreError: 141901:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 1 previous similar message [ 3944.594683] LustreError: 141880:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1787795168 with bad export cookie 53110103597116687 [ 3944.619380] LustreError: MGC192.168.203.127@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 3944.645836] LustreError: 141880:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 3945.282981] Lustre: server umount lustre-MDT0001 complete [ 3958.016090] Lustre: DEBUG MARKER: oleg327-server.virtnet: executing unload_modules_local [ 3960.495622] Key type lgssc unregistered [ 3960.960973] LNet: 145845:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 3960.972726] LNetError: 145845:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 3960.986208] LNet: Removed LNI 192.168.203.127@tcp [ 3961.945110] Key type .llcrypt unregistered [ 3961.947591] Key type ._llcrypt unregistered [ 3990.518738] Key type ._llcrypt registered [ 3990.523909] Key type .llcrypt registered [ 3990.672158] Lustre: DEBUG MARKER: oleg327-server.virtnet: executing load_modules_local [ 3991.612203] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 3991.636308] alg: No test for adler32 (adler32-zlib) [ 3992.837410] Lustre: Lustre: Build Version: 2.17.57_50_g504224c [ 3993.145026] LNet: Added LNI 192.168.203.127@tcp [8/256/0/180] [ 3994.849993] Key type lgssc registered [ 3996.007504] Lustre: Echo OBD driver; http://www.lustre.org/ [ 4005.027491] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 4005.074129] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 4006.686420] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 4011.608152] Lustre: DEBUG MARKER: oleg327-server.virtnet: executing set_default_debug -1 all [ 4016.109084] LustreError: 147261:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 4016.129740] LustreError: 147261:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 2 previous similar messages [ 4020.891582] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 4021.316304] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 4026.250093] Lustre: DEBUG MARKER: oleg327-server.virtnet: executing set_default_debug -1 all [ 4035.823773] Lustre: DEBUG MARKER: oleg327-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 4042.655525] Lustre: DEBUG MARKER: oleg327-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 4050.677862] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 4051.163730] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 4052.235292] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:85 to 0x280000401:161) [ 4058.531583] Lustre: DEBUG MARKER: oleg327-server.virtnet: executing set_default_debug -1 all [ 4066.543636] Lustre: lustre-OST0000: new connection from lustre-MDT0001-mdtlov (cleaning up unused objects from 0x280000400:72 to 0x280000400:129) [ 4067.077947] Lustre: DEBUG MARKER: oleg327-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 4073.000072] Lustre: *** cfs_fail_loc=159, val=0*** [ 4088.704995] Lustre: lustre-MDT0000: Client f00630ad-5948-4be5-8d05-77b2a142311a (at 192.168.203.27@tcp) reconnecting [ 4107.753204] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 4107.760540] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4107.766257] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 4107.777167] Lustre: Skipped 1 previous similar message [ 4107.796085] Lustre: Skipped 1 previous similar message [ 4112.538385] Lustre: server umount lustre-OST0000 complete [ 4117.996387] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4118.021676] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 4123.056484] Lustre: server umount lustre-MDT0000 complete [ 4123.107099] LustreError: 147261:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 4123.132840] LustreError: 147261:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 1 previous similar message [ 4127.021909] LustreError: 150176:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1787795351 with bad export cookie 1463557297587129517 [ 4127.036046] LustreError: MGC192.168.203.127@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 4127.041257] LustreError: 150176:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 3 previous similar messages [ 4127.367363] Lustre: server umount lustre-MDT0001 complete [ 4141.028268] Lustre: DEBUG MARKER: oleg327-server.virtnet: executing unload_modules_local [ 4144.410956] Key type lgssc unregistered [ 4144.770779] LNet: 150932:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 4144.786481] LNetError: 150932:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 4144.813365] LNet: Removed LNI 192.168.203.127@tcp [ 4145.794624] Key type .llcrypt unregistered [ 4145.797863] Key type ._llcrypt unregistered [ 4173.898034] Key type ._llcrypt registered [ 4173.905563] Key type .llcrypt registered [ 4174.062654] Lustre: DEBUG MARKER: oleg327-server.virtnet: executing load_modules_local [ 4175.440409] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 4175.554403] alg: No test for adler32 (adler32-zlib) [ 4176.664468] Lustre: Lustre: Build Version: 2.17.57_50_g504224c [ 4176.921807] LNet: Added LNI 192.168.203.127@tcp [8/256/0/180] [ 4178.608141] Key type lgssc registered [ 4179.559645] Lustre: Echo OBD driver; http://www.lustre.org/ [ 4188.290688] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 4188.322041] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 4189.867073] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 4194.759599] Lustre: DEBUG MARKER: oleg327-server.virtnet: executing set_default_debug -1 all [ 4199.399750] LustreError: 152349:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 4199.422260] LustreError: 152349:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 2 previous similar messages [ 4204.246736] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 4204.520865] LustreError: 152348:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 4205.182512] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 4211.487783] Lustre: DEBUG MARKER: oleg327-server.virtnet: executing set_default_debug -1 all [ 4220.674393] Lustre: DEBUG MARKER: oleg327-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 4227.517705] Lustre: DEBUG MARKER: oleg327-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 4235.080826] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 4235.620626] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 4242.062592] Lustre: DEBUG MARKER: oleg327-server.virtnet: executing set_default_debug -1 all [ 4249.939478] Lustre: DEBUG MARKER: oleg327-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 4249.979921] Lustre: lustre-OST0000: new connection from lustre-MDT0001-mdtlov (cleaning up unused objects from 0x280000400:72 to 0x280000400:161) [ 4259.187699] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:170 to 0x280000401:193) [ 4280.594572] Lustre: lustre-OST0000: evicting c9578025-764a-4292-b225-41c6c6bb2691 (at 192.168.203.27@tcp) by administrative request [ 4280.607730] Lustre: Skipped 1 previous similar message [ 4310.501344] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4310.501806] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 4310.505730] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 4310.518298] Lustre: Skipped 1 previous similar message [ 4315.510922] Lustre: server umount lustre-OST0000 complete [ 4320.750792] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4320.760256] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 4320.770603] Lustre: Skipped 1 previous similar message [ 4320.779105] Lustre: Skipped 2 previous similar messages [ 4325.845130] Lustre: server umount lustre-MDT0000 complete [ 4325.872747] LustreError: 152344:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 4325.911046] LustreError: 152344:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 1 previous similar message [ 4329.477809] LustreError: 152329:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1787795553 with bad export cookie 1800498439387528434 [ 4329.485564] LustreError: MGC192.168.203.127@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 4329.504509] LustreError: 152329:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 4329.973376] Lustre: server umount lustre-MDT0001 complete [ 4341.974633] Lustre: DEBUG MARKER: oleg327-server.virtnet: executing unload_modules_local [ 4344.645836] Key type lgssc unregistered [ 4344.978963] LNet: 156444:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 4344.989307] LNetError: 156444:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 4345.023870] LNet: Removed LNI 192.168.203.127@tcp [ 4345.940959] Key type .llcrypt unregistered [ 4345.943521] Key type ._llcrypt unregistered [ 4432.868331] Key type ._llcrypt registered [ 4432.870236] Key type .llcrypt registered [ 4433.044970] Lustre: DEBUG MARKER: oleg327-server.virtnet: executing load_modules_local [ 4434.220158] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 4434.243042] alg: No test for adler32 (adler32-zlib) [ 4435.284180] Lustre: Lustre: Build Version: 2.17.57_50_g504224c [ 4435.522384] LNet: Added LNI 192.168.203.127@tcp [8/256/0/180] [ 4437.272623] Key type lgssc registered [ 4439.565942] Lustre: Echo OBD driver; http://www.lustre.org/ [ 4450.961741] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 4450.988695] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 4452.714032] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 4458.193790] Lustre: DEBUG MARKER: oleg327-server.virtnet: executing set_default_debug -1 all [ 4462.053626] LustreError: 158942:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 4462.072612] LustreError: 158942:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 2 previous similar messages [ 4467.173376] LustreError: 158943:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 4468.281387] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 4468.755168] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 4474.348687] Lustre: DEBUG MARKER: oleg327-server.virtnet: executing set_default_debug -1 all [ 4482.897784] Lustre: DEBUG MARKER: oleg327-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 4490.220745] Lustre: DEBUG MARKER: oleg327-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 4501.142496] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 4501.611579] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 4510.611570] Lustre: DEBUG MARKER: oleg327-server.virtnet: executing set_default_debug -1 all [ 4513.930659] Lustre: lustre-OST0000: new connection from lustre-MDT0001-mdtlov (cleaning up unused objects from 0x280000400:72 to 0x280000400:193) [ 4519.354870] Lustre: DEBUG MARKER: oleg327-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 4522.105847] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:195 to 0x280000401:225) [ 4534.753503] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 4534.760491] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4534.796718] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 4537.827675] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 4537.836262] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 4537.837623] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4537.859308] Lustre: Skipped 1 previous similar message [ 4541.015696] Lustre: server umount lustre-OST0000 complete [ 4545.509851] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 4545.525110] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4545.547555] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 4548.073673] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 4548.074474] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4553.188059] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 4553.200603] Lustre: Skipped 2 previous similar messages [ 4559.329438] Lustre: lustre-MDT0000 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 4559.520205] Lustre: server umount lustre-MDT0000 complete [ 4562.411161] LustreError: 159674:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 4562.428537] LustreError: 159674:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 1 previous similar message [ 4563.650512] LustreError: 158921:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1787795787 with bad export cookie 8041045416402436507 [ 4563.654238] LustreError: MGC192.168.203.127@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 4563.665507] LustreError: 158921:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 3 previous similar messages [ 4564.200204] Lustre: server umount lustre-MDT0001 complete [ 4577.876880] Lustre: DEBUG MARKER: oleg327-server.virtnet: executing unload_modules_local [ 4580.667802] Key type lgssc unregistered [ 4580.965179] LNet: 162512:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 4580.972888] LNetError: 162512:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 4581.995711] LNet: Removed LNI 192.168.203.127@tcp [ 4583.041273] Key type .llcrypt unregistered [ 4583.043093] Key type ._llcrypt unregistered [ 4614.133160] Key type ._llcrypt registered [ 4614.135589] Key type .llcrypt registered [ 4614.300584] Lustre: DEBUG MARKER: oleg327-server.virtnet: executing load_modules_local [ 4615.658072] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 4615.856224] alg: No test for adler32 (adler32-zlib) [ 4617.129692] Lustre: Lustre: Build Version: 2.17.57_50_g504224c [ 4617.432866] LNet: Added LNI 192.168.203.127@tcp [8/256/0/180] [ 4619.232933] Key type lgssc registered [ 4620.339178] Lustre: Echo OBD driver; http://www.lustre.org/ [ 4637.891601] Lustre: DEBUG MARKER: == conf-sanity test 100: check lshowmount lists MGS, MDT, OST and 0@lo ========================================================== 21:57:40 (1787795860) [ 4657.435777] Lustre: DEBUG MARKER: oleg327-server.virtnet: executing set_hostid [ 4669.562808] Lustre: DEBUG MARKER: oleg327-server.virtnet: executing load_modules_local [ 4707.030187] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 4707.098927] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 4708.345884] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 4708.410533] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 4708.559879] Lustre: lustre-MDT0000: new disk, initializing [ 4708.690797] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 4708.714069] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 4713.556931] Lustre: DEBUG MARKER: oleg327-server.virtnet: executing set_default_debug -1 all [ 4723.849462] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 4724.025861] Lustre: 166773:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 4724.084623] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [ 4724.088576] Lustre: Skipped 1 previous similar message [ 4724.151809] Lustre: lustre-MDT0001: new disk, initializing [ 4724.246193] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 4724.328603] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [ 4724.347397] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 4728.814445] Lustre: DEBUG MARKER: oleg327-server.virtnet: executing set_default_debug -1 all [ 4738.253898] Lustre: DEBUG MARKER: oleg327-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 4744.344920] Lustre: DEBUG MARKER: oleg327-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 4751.582231] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 4751.897953] Lustre: lustre-OST0000: new disk, initializing [ 4751.903558] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 4751.913164] Lustre: 168744:0:(osd_compat.c:1353:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 4751.999975] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 4758.565633] Lustre: DEBUG MARKER: oleg327-server.virtnet: executing set_default_debug -1 all [ 4759.654332] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [ 4759.661907] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 4759.802070] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 4768.357147] Lustre: DEBUG MARKER: oleg327-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 4780.519419] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4780.539566] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 4780.542792] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 4780.547310] Lustre: Skipped 1 previous similar message [ 4784.482738] Lustre: server umount lustre-OST0000 complete [ 4790.754606] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4790.757282] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 4790.766583] Lustre: Skipped 2 previous similar messages [ 4790.783631] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 4794.962391] Lustre: server umount lustre-MDT0000 complete [ 4798.135030] LustreError: 166767:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1787796022 with bad export cookie 8494411197511525480 [ 4798.140819] LustreError: MGC192.168.203.127@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 4798.150725] LustreError: 166767:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 2 previous similar messages [ 4798.483663] Lustre: server umount lustre-MDT0001 complete [ 4809.123738] Lustre: DEBUG MARKER: oleg327-server.virtnet: executing unload_modules_local [ 4812.135198] Key type lgssc unregistered [ 4812.400797] LNet: 170654:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 4812.415232] LNetError: 170654:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 4812.428723] LNet: Removed LNI 192.168.203.127@tcp [ 4813.318658] Key type .llcrypt unregistered [ 4813.320745] Key type ._llcrypt unregistered [ 4840.447558] Key type ._llcrypt registered [ 4840.449471] Key type .llcrypt registered [ 4840.591971] Lustre: DEBUG MARKER: oleg327-server.virtnet: executing load_modules_local [ 4842.228465] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 4842.335076] alg: No test for adler32 (adler32-zlib) [ 4843.560211] Lustre: Lustre: Build Version: 2.17.57_50_g504224c [ 4843.904142] LNet: Added LNI 192.168.203.127@tcp [8/256/0/180] [ 4845.562850] Key type lgssc registered [ 4846.913186] Lustre: Echo OBD driver; http://www.lustre.org/ [ 4855.075486] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 4855.112334] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 4856.768871] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 4861.534994] Lustre: DEBUG MARKER: oleg327-server.virtnet: executing set_default_debug -1 all [ 4866.023321] LustreError: 172080:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 4866.042079] LustreError: 172080:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 2 previous similar messages [ 4870.994569] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 4871.137899] LustreError: 172099:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 4871.480548] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 4875.832422] Lustre: DEBUG MARKER: oleg327-server.virtnet: executing set_default_debug -1 all [ 4883.389877] Lustre: DEBUG MARKER: oleg327-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 4889.606333] Lustre: DEBUG MARKER: oleg327-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 4897.193773] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 4897.499510] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 4901.629130] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:3 to 0x280000401:33) [ 4904.495772] Lustre: DEBUG MARKER: oleg327-server.virtnet: executing set_default_debug -1 all [ 4912.003131] Lustre: DEBUG MARKER: oleg327-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 4922.326824] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 4922.367746] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4922.390521] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 4922.400702] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4922.421240] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 4923.353409] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 4923.389386] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4923.407257] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 4923.420242] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4923.442646] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 4924.294216] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4925.324244] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 4925.334712] Lustre: Skipped 1 previous similar message [ 4925.360544] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4925.369169] Lustre: Skipped 1 previous similar message [ 4925.371533] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 4925.375648] Lustre: Skipped 1 previous similar message [ 4925.378837] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4925.386401] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 4925.394892] Lustre: Skipped 1 previous similar message [ 4926.362525] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4927.257341] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4928.005953] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 4928.020027] Lustre: Skipped 2 previous similar messages [ 4928.043209] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4928.054389] Lustre: Skipped 2 previous similar messages [ 4928.061769] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 4928.066529] Lustre: Skipped 2 previous similar messages [ 4928.069888] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4928.081986] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 4928.092494] Lustre: Skipped 2 previous similar messages [ 4929.016559] LustreError: 172101:0:(osp_precreate.c:655:osp_precreate_send()) lustre-OST0000-osc-MDT0000: can't precreate: rc = -5 [ 4929.024953] LustreError: 172101:0:(osp_precreate.c:1359:osp_precreate_thread()) lustre-OST0000-osc-MDT0000: cannot precreate objects: rc = -5 [ 4929.045153] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4929.955447] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4930.736777] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4931.700428] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4932.810480] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 4932.823574] Lustre: Skipped 4 previous similar messages [ 4932.855955] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4932.869553] Lustre: Skipped 4 previous similar messages [ 4932.872994] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 4932.891409] Lustre: Skipped 4 previous similar messages [ 4932.911236] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4932.929929] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 4932.940743] Lustre: Skipped 4 previous similar messages [ 4933.902963] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4934.753535] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4935.653172] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4936.421279] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4937.235226] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4938.081078] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4938.785062] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4939.613471] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4940.380032] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4941.168473] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 4941.177782] Lustre: Skipped 9 previous similar messages [ 4941.197970] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4941.208786] Lustre: Skipped 9 previous similar messages [ 4941.213872] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 4941.219220] Lustre: Skipped 9 previous similar messages [ 4941.226722] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4941.243796] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 4941.248993] Lustre: Skipped 9 previous similar messages [ 4942.027745] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4942.856284] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4943.723787] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4944.530623] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4945.411046] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4946.298636] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4946.929495] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4947.894421] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4948.800771] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4949.615058] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4950.495780] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4951.497414] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4952.401480] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4953.380929] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4954.245196] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4955.108152] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4955.895546] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4956.939685] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4957.740704] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 4957.755312] Lustre: Skipped 18 previous similar messages [ 4957.775589] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4957.793192] Lustre: Skipped 18 previous similar messages [ 4957.798267] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 4957.807599] Lustre: Skipped 18 previous similar messages [ 4957.816780] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4957.838268] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 4957.847266] Lustre: Skipped 18 previous similar messages [ 4958.729472] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4959.579272] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4960.435689] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4961.227457] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4962.186977] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4963.064701] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4964.151803] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4965.021772] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4966.188577] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4967.061818] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4967.923621] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4968.678905] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4969.472956] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4970.303664] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4971.264341] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4972.224428] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4973.114726] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4973.998460] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4974.824173] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4975.731876] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4976.487252] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4977.314521] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4978.161428] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4978.923259] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4979.794209] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4980.577942] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4981.523419] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4982.329729] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4983.442451] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4984.485455] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4985.480624] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4986.351194] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4987.316867] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4988.110487] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4988.943880] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4989.712439] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4990.588827] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 4990.598293] Lustre: Skipped 36 previous similar messages [ 4990.623243] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4990.639072] Lustre: Skipped 36 previous similar messages [ 4990.645229] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 4990.650698] Lustre: Skipped 36 previous similar messages [ 4990.654367] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4990.696369] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 4990.699700] Lustre: Skipped 36 previous similar messages [ 4991.606113] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4992.441854] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4993.363485] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4994.231220] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4995.028434] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4995.894400] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4996.794281] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4997.493484] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4998.419585] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4999.283995] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5000.126413] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5001.014933] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5001.895821] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5002.668689] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5003.512790] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5004.348438] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5005.465052] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5006.260383] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5007.450422] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5008.391847] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5009.421718] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5010.245386] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5011.168447] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5012.242283] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5013.268889] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5014.327317] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5015.397940] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5016.370683] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5017.339818] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5018.290178] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5019.557958] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5020.578409] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5021.738571] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5022.713825] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5024.124051] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5024.776229] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5025.710376] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5026.709414] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5027.749949] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5028.526826] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5029.794072] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5030.983927] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5032.183206] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5033.455048] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5034.765917] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5035.655987] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5036.471154] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5037.104476] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5037.879658] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5038.676622] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5039.598818] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5040.517712] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5041.660589] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5042.624444] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5043.870702] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5044.862243] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5045.773308] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5046.630615] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5047.463542] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5048.233072] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5049.188966] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5049.959177] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5050.960069] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5051.962896] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5053.005288] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5053.853779] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5054.627966] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 5054.631655] Lustre: Skipped 66 previous similar messages [ 5054.665893] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5054.674877] Lustre: Skipped 66 previous similar messages [ 5054.678383] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 5054.683296] Lustre: Skipped 66 previous similar messages [ 5054.687376] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5055.492035] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5055.499440] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 5055.505808] Lustre: Skipped 67 previous similar messages [ 5056.322572] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5057.202877] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5058.312896] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5059.108218] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5060.214041] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5061.029211] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5062.062597] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5062.960783] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5063.994836] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5064.881586] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5065.796886] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5066.637806] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5067.492644] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5068.761794] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5069.863291] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5070.763499] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5071.639971] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5072.595099] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5073.560524] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5074.423202] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5075.350765] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5076.170710] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5077.165147] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5078.108812] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5078.879885] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5079.662869] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5080.529290] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5081.291857] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5082.111336] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5082.860848] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5083.702438] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5084.385806] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5085.147081] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5085.974428] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5086.886846] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5087.576348] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5088.350391] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5089.171780] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5090.028798] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5090.792655] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5091.562979] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5092.361921] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5093.076642] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5093.794955] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5094.575127] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5095.445060] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5096.321871] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5097.206459] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5098.075140] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5098.917320] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5099.943334] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5100.770454] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5101.725523] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5102.487567] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5103.344356] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5104.372741] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5105.215750] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5106.150895] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5107.069274] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5108.031530] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5109.246666] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5110.194332] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5111.137525] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5112.041793] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5113.033520] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5114.043094] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5115.060598] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5115.840887] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5116.719562] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5117.529634] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5118.329173] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5119.031207] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5119.802072] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5120.509012] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5121.345396] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5122.198571] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5123.125372] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5123.936360] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5124.711078] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5125.533423] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5126.322602] LustreError: 172101:0:(osp_precreate.c:655:osp_precreate_send()) lustre-OST0000-osc-MDT0000: can't precreate: rc = -5 [ 5126.331468] LustreError: 172101:0:(osp_precreate.c:1359:osp_precreate_thread()) lustre-OST0000-osc-MDT0000: cannot precreate objects: rc = -5 [ 5126.336314] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5127.110808] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5127.818921] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5128.631486] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5129.307057] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5130.040415] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5130.770174] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5131.680064] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5132.625573] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5133.419964] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5134.138637] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5134.903564] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5135.729448] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5136.668931] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5137.565836] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5138.322084] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5139.072046] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5139.793637] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5140.554750] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5141.249347] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5142.004941] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5142.721847] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5143.632820] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5144.362404] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5145.213384] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5145.991705] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5146.852852] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5147.598540] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5148.405273] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5149.087141] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5149.904317] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5150.800303] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5151.736322] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5152.498296] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5153.421884] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5154.243559] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5155.209589] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5156.084548] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5156.884900] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5157.721268] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5158.660034] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5159.402143] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5160.336937] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5161.154474] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5162.170213] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5162.926615] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5163.823378] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5164.659920] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5165.689488] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5166.544218] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5167.710502] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5168.715979] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5169.932163] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5170.717859] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5171.944613] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5172.891801] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5174.145953] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5174.959799] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5175.999534] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5176.936987] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5177.971474] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5179.213752] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5180.348348] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5181.144367] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5181.947830] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5182.801957] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 5182.810263] Lustre: Skipped 146 previous similar messages [ 5182.836086] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5182.852494] Lustre: Skipped 146 previous similar messages [ 5182.870030] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 5182.884837] Lustre: Skipped 146 previous similar messages [ 5182.897454] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5183.781444] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5183.805947] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 5183.809281] Lustre: Skipped 146 previous similar messages [ 5184.684050] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5185.607453] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5186.497908] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5187.296985] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5188.225938] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5189.246608] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5190.236914] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5191.319265] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5192.164332] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5193.230052] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5194.171835] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5195.240489] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5196.147881] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5196.980277] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5197.812885] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5198.726618] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5199.724879] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5200.711853] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5201.694698] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5202.483127] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5203.235475] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5204.160557] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5205.032320] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5206.046305] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5206.893598] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5207.654782] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5208.339281] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5209.094967] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5209.743870] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5210.528976] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5211.271290] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5212.083438] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5212.974281] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5213.835213] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5214.680880] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5215.596097] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5216.452806] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5217.421616] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5218.238383] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5219.260532] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5220.247293] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5221.049120] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5221.951132] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5223.028332] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5223.881763] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5224.779914] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5225.672335] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5226.589463] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5227.428679] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5228.457898] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5229.430574] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5230.394075] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5231.236860] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5232.195306] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5232.980968] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5233.927503] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5234.660967] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5235.508474] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5236.284809] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5237.159859] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5238.050643] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5238.839771] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5239.653329] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5240.491398] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5241.415041] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5242.286254] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5242.943575] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5243.772718] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5244.438063] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5245.244462] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5246.015075] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5246.923194] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5247.740793] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5248.533176] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5249.277293] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5250.138183] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5250.988708] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5251.809329] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5252.629190] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5253.648692] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5254.603265] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5255.609679] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5256.590701] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5257.557891] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5258.261404] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5259.107550] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5260.046972] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5260.946086] LustreError: 172101:0:(osp_precreate.c:655:osp_precreate_send()) lustre-OST0000-osc-MDT0000: can't precreate: rc = -5 [ 5260.966262] LustreError: 172101:0:(osp_precreate.c:1359:osp_precreate_thread()) lustre-OST0000-osc-MDT0000: cannot precreate objects: rc = -5 [ 5260.968761] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5261.758486] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5262.622803] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5263.579210] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5264.479649] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5265.284139] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5266.235180] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5267.108934] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5268.116200] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5269.059420] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5270.126340] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5270.981615] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5271.856170] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5272.759959] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5273.723756] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5275.052231] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5275.929860] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5276.665697] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5277.632067] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5278.485422] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5279.310889] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5280.324110] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5281.169331] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5282.188419] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5283.247370] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5284.442265] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5285.872316] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5287.236213] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5288.247817] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5289.397371] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5290.529589] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5291.520213] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5292.457292] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5293.479122] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5294.793322] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5296.116247] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5297.366231] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5298.326354] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5299.024105] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5299.862374] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5301.038496] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5301.783035] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5302.667049] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5303.519795] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5304.373512] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5305.106931] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5305.958419] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5306.737228] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5307.781497] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5308.677109] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5309.614121] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5310.527310] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5311.443281] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5312.352670] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5313.424283] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5314.561298] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5315.639634] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5316.676640] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5317.702394] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5318.711334] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5319.759901] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5320.661483] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5321.491166] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5322.352852] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5323.058327] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5323.818218] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5324.678030] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5325.585825] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5326.351942] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5327.121507] LustreError: 172101:0:(osp_precreate.c:655:osp_precreate_send()) lustre-OST0000-osc-MDT0000: can't precreate: rc = -5 [ 5327.135161] LustreError: 172101:0:(osp_precreate.c:1359:osp_precreate_thread()) lustre-OST0000-osc-MDT0000: cannot precreate objects: rc = -5 [ 5327.143843] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5327.975482] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5328.698357] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5329.416907] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5330.112230] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5330.891439] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5331.696111] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5332.447652] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5333.309425] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5334.220807] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5335.041140] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5336.091721] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5336.896446] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5338.005391] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5338.894078] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5340.073644] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5341.046961] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5342.107257] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5343.160282] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5344.089114] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5345.209172] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5346.496346] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5347.703377] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5348.809763] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5349.862632] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5350.947351] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5352.172499] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5353.279218] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5354.428157] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5355.751704] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5356.718758] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5357.772996] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5359.062847] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5360.170554] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5361.076799] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5361.992613] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5362.786374] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5363.669264] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5364.533819] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5365.402081] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5366.313929] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5367.249273] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5368.169970] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5369.044615] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5369.927679] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5370.763204] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5371.541201] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5372.347212] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5373.128388] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5373.995033] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5374.709245] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5375.642420] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5376.463952] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5377.234951] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5378.160644] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5379.185857] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5380.158520] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5381.139558] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5382.100723] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5383.088500] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5384.020798] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5385.086656] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5386.234739] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5387.197941] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5388.392259] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5389.319498] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5390.525042] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5391.600481] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5392.716942] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5393.871948] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5394.918228] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5396.061385] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5397.328921] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5398.457700] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5399.797314] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5400.784251] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5401.698862] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5402.857543] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5403.618991] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5404.683475] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5405.498661] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5406.636559] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5407.792546] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5408.637094] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5409.579285] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5410.436690] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5411.264375] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5412.085432] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5413.196482] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5414.402663] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5415.439433] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5416.534855] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5417.569291] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5418.671427] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5419.524276] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5420.412159] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5421.364943] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5422.648739] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5423.610512] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5424.491505] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5425.348533] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5426.308861] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5427.279968] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5428.169252] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5429.107501] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5430.066190] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5431.028552] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5432.236977] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5433.241518] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5434.218217] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5435.233372] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5436.427314] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5437.369414] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5438.471436] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5439.310587] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 5439.315074] Lustre: Skipped 271 previous similar messages [ 5439.328819] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5439.343755] Lustre: Skipped 271 previous similar messages [ 5439.354116] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 5439.363647] Lustre: Skipped 271 previous similar messages [ 5439.367979] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5440.394932] LustreError: 172101:0:(osp_precreate.c:655:osp_precreate_send()) lustre-OST0000-osc-MDT0000: can't precreate: rc = -5 [ 5440.406335] LustreError: 172101:0:(osp_precreate.c:1359:osp_precreate_thread()) lustre-OST0000-osc-MDT0000: cannot precreate objects: rc = -5 [ 5440.410141] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5440.444552] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 5440.457957] Lustre: Skipped 271 previous similar messages [ 5441.388273] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5442.262932] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5443.339050] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5444.304773] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5445.412858] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5446.341769] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5447.310531] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5448.180875] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5449.035515] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5449.857187] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5450.916889] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5451.930625] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5453.220423] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5454.028866] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5455.432530] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5456.917943] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5458.138912] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5459.396695] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5461.049268] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5462.347840] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5463.656689] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5464.930764] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5466.155596] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5467.175062] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5468.552679] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5470.044491] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5471.589795] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5472.750538] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5474.022660] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5474.868167] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5475.796831] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5476.799931] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5477.782565] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5478.787878] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5479.988550] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5481.132060] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5482.277495] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5483.304406] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5484.245135] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5485.087555] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5486.061386] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5486.945763] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5487.944335] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5488.818721] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5489.828150] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5490.900947] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5491.664930] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5492.567990] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5493.465651] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5494.489061] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5495.371193] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5496.359408] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5497.311573] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5498.180194] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5499.087222] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5500.060090] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5500.960481] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5501.707609] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5502.488141] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5503.255836] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5504.150796] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5504.905470] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5505.769815] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5506.604672] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5507.462833] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5508.299127] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5509.170043] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5510.048139] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5510.842301] LustreError: 172101:0:(osp_precreate.c:655:osp_precreate_send()) lustre-OST0000-osc-MDT0000: can't precreate: rc = -5 [ 5510.848534] LustreError: 172101:0:(osp_precreate.c:1359:osp_precreate_thread()) lustre-OST0000-osc-MDT0000: cannot precreate objects: rc = -5 [ 5510.856952] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5511.665485] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5512.539516] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5513.418503] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5514.305490] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5515.087824] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5515.965887] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5516.828393] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5517.818136] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5518.590782] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5519.452305] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5520.335313] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5521.181873] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5522.073572] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5523.128639] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5523.880255] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5524.791709] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5525.701594] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5526.698184] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5527.540995] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5528.420560] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5529.642589] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5530.802494] LustreError: 172101:0:(osp_precreate.c:655:osp_precreate_send()) lustre-OST0000-osc-MDT0000: can't precreate: rc = -5 [ 5530.811832] LustreError: 172101:0:(osp_precreate.c:1359:osp_precreate_thread()) lustre-OST0000-osc-MDT0000: cannot precreate objects: rc = -5 [ 5530.842541] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5532.060807] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5532.897381] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5533.817175] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5534.723287] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5535.786906] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5536.796848] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5537.832273] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5538.757104] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5539.696154] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5540.676726] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5541.689817] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5542.662399] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5543.687827] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5544.691759] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5545.659389] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5546.721709] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5547.707625] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5548.497381] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5549.435048] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5550.428237] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5551.250549] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5552.361179] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5553.226216] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5554.114313] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5555.264754] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5556.278048] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5557.427051] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5558.571196] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5559.750212] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5560.694253] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5561.802661] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5562.651388] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5563.551331] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5564.494905] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5565.489408] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5566.515596] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5567.610933] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5568.780466] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5569.690541] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5570.733401] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5571.748516] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5572.693223] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5573.517162] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5574.460892] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5575.564469] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5576.543555] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5577.946837] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5579.166240] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5580.155875] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5581.197227] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5582.368866] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5583.447487] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5584.621927] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5585.728898] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5587.058279] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5588.395401] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5589.833968] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5590.787992] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5592.125848] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5593.319629] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5594.614540] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5595.734258] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5596.758365] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5597.991827] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5599.071758] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5599.965894] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5601.142630] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5602.193023] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5603.189975] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5604.314578] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5605.196885] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5606.170720] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5607.012727] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5608.014426] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5608.903644] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5609.788795] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5610.662811] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5611.608833] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5612.466431] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5613.361898] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5614.285915] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5615.210748] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5616.202073] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5617.150905] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5618.243238] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5619.269076] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5620.215571] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5621.048148] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5622.075043] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5623.057302] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5623.961487] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5624.879888] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5625.690919] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5626.570110] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5627.510049] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5628.325853] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5629.156641] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5630.012255] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5630.983553] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5631.908693] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5632.798132] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5633.656532] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5634.483799] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5635.361831] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5636.216656] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5637.094050] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5638.179272] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5639.154441] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5640.094851] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5641.129702] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5642.229586] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5643.426562] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5644.338866] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5645.332345] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5646.497481] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5647.650857] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5648.880177] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5649.786118] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5650.820059] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5651.735690] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5652.781672] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5654.031538] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5655.192979] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5656.334916] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5657.316208] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5658.321499] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5659.161560] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5660.181160] LustreError: 172101:0:(osp_precreate.c:655:osp_precreate_send()) lustre-OST0000-osc-MDT0000: can't precreate: rc = -5 [ 5660.202134] LustreError: 172101:0:(osp_precreate.c:1359:osp_precreate_thread()) lustre-OST0000-osc-MDT0000: cannot precreate objects: rc = -5 [ 5660.204138] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5661.384521] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5662.303979] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5663.195590] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5663.975889] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5664.880347] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5665.814198] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5666.757258] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5667.825165] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5668.725466] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5669.731020] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5670.646469] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5671.669366] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5672.663788] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5673.625573] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5674.941699] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5676.058248] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5677.089727] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5678.073921] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5678.950563] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5679.877064] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5680.949254] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5681.919323] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5682.941668] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5684.047024] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5685.112221] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5686.073384] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5687.137703] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5688.023377] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5689.030172] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5689.928463] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5690.898841] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5691.622325] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5692.418377] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5693.215836] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5694.347116] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5695.417144] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5696.717132] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5697.836884] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5698.847530] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5700.103574] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5701.112904] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5702.183927] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5703.194447] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5704.057600] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5705.103747] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5706.464362] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5707.481408] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5708.696048] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5709.868929] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5711.067182] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5712.191282] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5713.232157] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5714.164315] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5715.242288] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5716.239237] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5717.179730] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5718.250524] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5719.365221] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5720.361177] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5721.525639] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5722.678438] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5723.628478] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5724.513951] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5725.424520] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5726.298844] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5727.317091] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5728.377656] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5729.416295] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5730.366132] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5731.323871] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5732.167835] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5733.178486] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5734.128929] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5735.174338] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5735.975326] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5736.972283] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5737.955685] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5739.024597] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5739.993218] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5741.014204] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5742.045312] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5743.023294] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5743.916875] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5744.881274] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5745.856291] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5746.870387] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5747.879900] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5748.744718] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5749.647084] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5750.681335] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5751.777427] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5752.759565] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5753.738274] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5754.813398] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5755.847518] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5756.728861] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5757.871888] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5758.743462] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5759.506036] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5760.519896] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5761.426205] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5762.483350] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5763.391462] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5764.247638] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5765.273022] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5766.124343] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5767.159520] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5768.148569] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5769.032389] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5769.868433] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5770.699180] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5771.496274] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5772.304454] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5773.123403] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5774.219578] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5775.153580] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5776.061373] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5776.919744] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5777.805070] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5778.640262] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5779.424168] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5780.350935] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5781.326603] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5782.262978] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5783.157449] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5783.996266] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5785.053582] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5785.963166] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5787.173857] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5788.191456] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5789.083575] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5789.959139] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5790.782296] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5791.639822] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5792.481991] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5793.318570] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5794.184960] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5795.137825] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5796.013245] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5796.904670] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5797.797241] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5798.730190] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5799.695663] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5800.609792] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5801.661441] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5802.491701] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5803.598284] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5804.490963] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5805.535144] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5806.442837] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5807.275104] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5808.269137] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5809.127467] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5810.026482] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5810.961572] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5812.204743] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5813.360741] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5814.437261] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5815.403989] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5816.303267] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5817.233840] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5818.242351] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5818.995964] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5819.982552] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5820.944296] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5821.897763] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5822.772580] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5823.806602] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5825.000822] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5826.082951] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5827.179143] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5828.092464] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5829.206583] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5830.359160] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5831.436311] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5832.704164] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5834.086669] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5835.371169] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5836.479136] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5837.773151] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5838.862914] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5839.847813] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5840.749301] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5841.958879] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5843.046579] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5844.412347] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5845.643269] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5846.905442] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5847.859380] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5848.887594] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5849.755284] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5850.749884] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5851.597429] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5852.521554] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5853.476550] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5854.358490] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5855.141161] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5856.076900] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5856.955692] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5857.954036] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5858.838949] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5859.731348] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5860.377576] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5861.178285] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5861.963352] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5862.847472] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5863.846886] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5864.840372] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5865.912463] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5866.752082] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5867.753925] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5868.630937] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5869.614954] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5870.498411] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5871.374625] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5872.280632] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6296.092171] LustreError: 174099:0:(ofd_dev.c:1780:ofd_create_hdl()) lustre-OST0000: unable to precreate: rc = -28 [ 6296.110524] LustreError: 172101:0:(osp_precreate.c:655:osp_precreate_send()) lustre-OST0000-osc-MDT0000: can't precreate: rc = -28 [ 6987.234360] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 6987.239490] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 6987.245504] Lustre: Skipped 436 previous similar messages [ 6987.248463] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 6987.750139] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 6992.867638] LustreError: 174099:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-OST0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 6992.882646] LustreError: 174099:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 1 previous similar message [ 6992.938257] Lustre: server umount lustre-OST0000 complete [ 6997.987892] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 6997.996404] Lustre: Skipped 1 previous similar message [ 7003.107853] LustreError: 172099:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 7003.142464] LustreError: 172099:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 1 previous similar message [ 7003.236333] Lustre: server umount lustre-MDT0000 complete [ 7006.998093] LustreError: 172060:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1787798231 with bad export cookie 10596584184016343473 [ 7007.009676] LustreError: 172060:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 2 previous similar messages [ 7007.013889] LustreError: MGC192.168.203.127@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 7007.582698] Lustre: server umount lustre-MDT0001 complete [ 7021.859790] Lustre: DEBUG MARKER: oleg327-server.virtnet: executing unload_modules_local [ 7025.412201] Key type lgssc unregistered [ 7025.977760] LNet: 227349:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 7025.986297] LNetError: 227349:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 7026.012217] LNet: Removed LNI 192.168.203.127@tcp [ 7027.371075] Key type .llcrypt unregistered [ 7027.372697] Key type ._llcrypt unregistered [ 7053.974561] Key type ._llcrypt registered [ 7053.977402] Key type .llcrypt registered [ 7054.126407] Lustre: DEBUG MARKER: oleg327-server.virtnet: executing load_modules_local [ 7055.172698] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 7055.270079] alg: No test for adler32 (adler32-zlib) [ 7056.419414] Lustre: Lustre: Build Version: 2.17.57_50_g504224c [ 7056.722372] LNet: Added LNI 192.168.203.127@tcp [8/256/0/180] [ 7058.384181] Key type lgssc registered [ 7059.827500] Lustre: Echo OBD driver; http://www.lustre.org/ [ 7067.649887] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 7067.670716] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 7069.139266] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 7073.385494] Lustre: DEBUG MARKER: oleg327-server.virtnet: executing set_default_debug -1 all [ 7079.411230] LustreError: 228765:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 7079.434295] LustreError: 228765:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 2 previous similar messages [ 7081.964458] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 7082.288681] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 7086.354947] Lustre: DEBUG MARKER: oleg327-server.virtnet: executing set_default_debug -1 all [ 7092.957844] Lustre: DEBUG MARKER: oleg327-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 7098.971203] Lustre: DEBUG MARKER: oleg327-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 7106.817150] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 7107.042180] LustreError: 230590:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-OST0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 7107.168701] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 7114.234262] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:49665 to 0x280000401:49696) [ 7115.180341] Lustre: DEBUG MARKER: oleg327-server.virtnet: executing set_default_debug -1 all [ 7123.878348] Lustre: DEBUG MARKER: oleg327-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 7129.122470] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 7131.087696] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 7131.099828] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 7131.104855] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7131.121788] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 7131.121909] LustreError: 228785:0:(osp_precreate.c:1268:osp_precreate_thread()) cfs_fail_timeout id 2107 sleeping for 20000ms [ 7134.697635] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 7134.706566] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 7134.719345] Lustre: Skipped 1 previous similar message [ 7134.751728] Lustre: Skipped 1 previous similar message [ 7136.056219] Lustre: lustre-OST0000: Not available for connect from 192.168.203.27@tcp (stopping) [ 7139.669137] Lustre: server umount lustre-OST0000 complete [ 7151.192090] LustreError: 228785:0:(osp_precreate.c:1268:osp_precreate_thread()) cfs_fail_timeout id 2107 awake [ 7176.050874] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 7176.457630] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 7182.709731] Lustre: DEBUG MARKER: oleg327-server.virtnet: executing set_default_debug -1 all [ 7189.938193] Lustre: DEBUG MARKER: oleg327-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 7199.992401] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7200.027590] LustreError: lustre-OST0000-osc-MDT0001: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7200.038691] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 7200.056133] Lustre: Skipped 1 previous similar message [ 7202.766659] Lustre: DEBUG MARKER: osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid in FULL state after 10 sec [ 7208.597886] Lustre: DEBUG MARKER: oleg327-client.virtnet: executing wait_import_state (FULL|IDLE) osc.lustre-OST0000-osc-ffff9afec7c62000.ost_server_uuid 50 [ 7210.421356] Lustre: DEBUG MARKER: osc.lustre-OST0000-osc-ffff9afec7c62000.ost_server_uuid in FULL state after 0 sec [ 7215.589994] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 7215.597506] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 7215.619247] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 7215.640679] Lustre: Skipped 1 previous similar message [ 7217.660542] Lustre: server umount lustre-OST0000 complete [ 7225.837547] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 7225.840538] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 7225.864387] Lustre: Skipped 1 previous similar message [ 7225.865625] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 7229.484559] Lustre: server umount lustre-MDT0000 complete [ 7230.945932] LustreError: 228762:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 7230.966303] LustreError: 228762:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 1 previous similar message [ 7233.825267] LustreError: 228744:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1787798457 with bad export cookie 14789359203743816050 [ 7233.830161] LustreError: MGC192.168.203.127@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 7233.846724] LustreError: 228744:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 2 previous similar messages [ 7234.302348] Lustre: server umount lustre-MDT0001 complete [ 7245.173037] Lustre: DEBUG MARKER: oleg327-server.virtnet: executing unload_modules_local [ 7247.807567] Key type lgssc unregistered [ 7248.027654] LNet: 233999:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 7248.039471] LNetError: 233999:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 7248.072573] LNet: Removed LNI 192.168.203.127@tcp [ 7248.907538] Key type .llcrypt unregistered [ 7248.910488] Key type ._llcrypt unregistered [ 7280.654329] Key type ._llcrypt registered [ 7280.661175] Key type .llcrypt registered [ 7280.817813] Lustre: DEBUG MARKER: oleg327-server.virtnet: executing load_modules_local [ 7281.808836] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 7281.853102] alg: No test for adler32 (adler32-zlib) [ 7283.046705] Lustre: Lustre: Build Version: 2.17.57_50_g504224c [ 7283.354908] LNet: Added LNI 192.168.203.127@tcp [8/256/0/180] [ 7285.032189] Key type lgssc registered [ 7286.675171] Lustre: Echo OBD driver; http://www.lustre.org/ [ 7296.634122] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 7296.652595] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 7298.026586] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 7302.519652] Lustre: DEBUG MARKER: oleg327-server.virtnet: executing set_default_debug -1 all [ 7308.260438] LustreError: 235737:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 7308.282422] LustreError: 235737:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 2 previous similar messages [ 7311.671600] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 7312.086380] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 7316.933247] Lustre: DEBUG MARKER: oleg327-server.virtnet: executing set_default_debug -1 all [ 7320.006346] Lustre: Modifying parameter general.debug_raw_pointers=Y in log params [ 7327.012225] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 7327.388532] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 7333.777153] Lustre: DEBUG MARKER: oleg327-server.virtnet: executing set_default_debug -1 all [ 7343.806783] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:49698 to 0x280000401:49728) [ 7344.021692] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 7344.214718] Lustre: lustre-OST0001: new disk, initializing [ 7344.218667] Lustre: srv-lustre-OST0001: No data found on store. Initialize space. [ 7344.224325] Lustre: 238044:0:(osd_compat.c:1353:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 7344.333406] Lustre: lustre-OST0001: Imperative Recovery not enabled, recovery window 60-180 [ 7349.915779] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x00000002c0000400-0x0000000300000400]:1:ost [ 7349.937769] Lustre: cli-lustre-OST0001-super: Allocated super-sequence [0x00000002c0000400-0x0000000300000400]:1:ost] [ 7350.144787] Lustre: lustre-OST0001-osc-MDT0000: update sequence from 0x100010000 to 0x2c0000401 [ 7351.914569] Lustre: DEBUG MARKER: oleg327-server.virtnet: executing set_default_debug -1 all [ 7364.118862] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 7375.250316] Lustre: Setting parameter general.lod.*.mdt_hash=crush in log params [ 7389.158044] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 7389.169585] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 7389.196317] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 7391.208254] Lustre: lustre-MDT0000-lwp-OST0000: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 7391.221871] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 7391.236975] Lustre: Skipped 1 previous similar message [ 7391.256757] Lustre: Skipped 3 previous similar messages [ 7395.608781] Lustre: server umount lustre-MDT0000 complete [ 7396.330609] LustreError: 235731:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 7396.346035] LustreError: 235731:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 3 previous similar messages [ 7399.877279] LustreError: 239045:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1787798623 with bad export cookie 14966718186513068519 [ 7399.892516] LustreError: MGC192.168.203.127@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 7400.316592] Lustre: server umount lustre-MDT0001 complete [ 7414.938358] Lustre: server umount lustre-OST0000 complete [ 7417.632222] Lustre: 234999:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1787798625/real 1787798625] req@ffff92b09816ad00 x1874642607375232/t0(0) o400->lustre-MDT0001-lwp-OST0001@0@lo:12/10 lens 224/224 e 0 to 1 dl 1787798641 ref 1 fl Rpc:XNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 [ 7417.688621] Lustre: lustre-MDT0001-lwp-OST0001: Connection to lustre-MDT0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 7417.708712] Lustre: Skipped 1 previous similar message [ 7419.936140] Lustre: server umount lustre-OST0001 complete [ 7444.636504] Lustre: DEBUG MARKER: oleg327-server.virtnet: executing load_modules_local [ 7457.471578] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 7458.158611] LustreError: 241156:0:(ldlm_lib.c:1190:target_handle_connect()) mylustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 7458.276148] Lustre: mylustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 7463.396253] LustreError: 241157:0:(ldlm_lib.c:1190:target_handle_connect()) mylustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 7464.822173] Lustre: DEBUG MARKER: oleg327-server.virtnet: executing set_default_debug -1 all [ 7468.520702] LustreError: 241156:0:(ldlm_lib.c:1190:target_handle_connect()) mylustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 7474.603306] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 7475.441397] Lustre: mylustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 7480.478188] Lustre: DEBUG MARKER: oleg327-server.virtnet: executing set_default_debug -1 all [ 7484.087272] Lustre: 242300:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify general/debug_raw_pointers=Y (mode = 0) failed: rc = -17 [ 7491.609937] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 7491.946695] Lustre: mylustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 7498.709530] Lustre: DEBUG MARKER: oleg327-server.virtnet: executing set_default_debug -1 all [ 7500.142417] LustreError: 242655:0:(ldlm_lib.c:1190:target_handle_connect()) mylustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 7500.165930] LustreError: 242655:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 1 previous similar message [ 7500.183713] Lustre: mylustre-OST0000: new connection from mylustre-MDT0001-mdtlov (cleaning up unused objects from 0x280000400:99 to 0x280000400:129) [ 7503.248879] Lustre: mylustre-OST0000: new connection from mylustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:49698 to 0x280000401:49760) [ 7507.779124] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 7513.625664] Lustre: mylustre-OST0001: new connection from mylustre-MDT0001-mdtlov (cleaning up unused objects from 0x2c0000400:3 to 0x2c0000400:33) [ 7514.901104] Lustre: DEBUG MARKER: oleg327-server.virtnet: executing set_default_debug -1 all [ 7523.109781] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 7526.772835] Lustre: 244169:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify general/lod.*.mdt_hash=crush (mode = 0) failed: rc = -17 [ 7539.180521] Lustre: mylustre-MDT0000-lwp-OST0001: Connection to mylustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 7539.191073] Lustre: mylustre-MDT0000: Not available for connect from 0@lo (stopping) [ 7539.205348] Lustre: Skipped 2 previous similar messages [ 7539.225800] Lustre: Skipped 3 previous similar messages [ 7543.150658] Lustre: server umount mylustre-MDT0000 complete [ 7544.291316] LustreError: 241156:0:(ldlm_lib.c:1190:target_handle_connect()) mylustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 7544.308825] LustreError: 241156:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 3 previous similar messages [ 7546.964701] LustreError: 241131:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1787798770 with bad export cookie 14966718186513071382 [ 7546.974662] LustreError: MGC192.168.203.127@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 7546.980419] LustreError: 241131:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 5 previous similar messages [ 7547.407587] Lustre: server umount mylustre-MDT0001 complete [ 7560.247447] Lustre: server umount mylustre-OST0000 complete [ 7573.974654] Lustre: server umount mylustre-OST0001 complete [ 7595.589072] Lustre: DEBUG MARKER: oleg327-server.virtnet: executing load_modules_local [ 7605.454090] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 7605.947776] LustreError: 246477:0:(ldlm_lib.c:1190:target_handle_connect()) tfs-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 7606.012697] Lustre: tfs-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 7606.023984] Lustre: Skipped 1 previous similar message [ 7610.808691] Lustre: DEBUG MARKER: oleg327-server.virtnet: executing set_default_debug -1 all [ 7620.397140] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 7626.389406] Lustre: DEBUG MARKER: oleg327-server.virtnet: executing set_default_debug -1 all [ 7630.034100] Lustre: 247620:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify general/debug_raw_pointers=Y (mode = 0) failed: rc = -17 [ 7637.289121] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 7645.436304] Lustre: DEBUG MARKER: oleg327-server.virtnet: executing set_default_debug -1 all [ 7645.944992] Lustre: tfs-OST0000: new connection from tfs-MDT0001-mdtlov (cleaning up unused objects from 0x280000400:99 to 0x280000400:161) [ 7650.044916] Lustre: tfs-OST0000: new connection from tfs-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:49698 to 0x280000401:49792) [ 7654.711255] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 7660.038922] Lustre: tfs-OST0001: new connection from tfs-MDT0001-mdtlov (cleaning up unused objects from 0x2c0000400:35 to 0x2c0000400:65) [ 7661.576308] Lustre: DEBUG MARKER: oleg327-server.virtnet: executing set_default_debug -1 all [ 7670.812334] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 7674.540567] Lustre: 249493:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify general/lod.*.mdt_hash=crush (mode = 0) failed: rc = -17 [ 7682.530098] LustreError: tfs-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 7682.540375] Lustre: tfs-MDT0000-osp-MDT0001: Connection to tfs-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 7682.559328] Lustre: Skipped 1 previous similar message [ 7682.571278] Lustre: tfs-MDT0000: Not available for connect from 0@lo (stopping) [ 7690.537796] Lustre: server umount tfs-MDT0000 complete [ 7690.733862] LustreError: 248245:0:(ldlm_lib.c:1190:target_handle_connect()) tfs-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 7690.763563] LustreError: 248245:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 9 previous similar messages [ 7694.654626] LustreError: 247998:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1787798918 with bad export cookie 14966718186513073559 [ 7694.663281] LustreError: MGC192.168.203.127@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 7694.677414] LustreError: 247998:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 5 previous similar messages [ 7695.085911] Lustre: server umount tfs-MDT0001 complete [ 7709.178339] Lustre: server umount tfs-OST0000 complete [ 7711.201076] Lustre: 234999:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1787798919/real 1787798919] req@ffff92b0856083c0 x1874642607490816/t0(0) o400->tfs-MDT0001-lwp-OST0001@0@lo:12/10 lens 224/224 e 0 to 1 dl 1787798935 ref 1 fl Rpc:XNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 [ 7711.245928] Lustre: tfs-MDT0001-lwp-OST0001: Connection to tfs-MDT0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 7711.257300] Lustre: Skipped 3 previous similar messages [ 7714.275168] Lustre: server umount tfs-OST0001 complete [ 7739.172068] Lustre: DEBUG MARKER: oleg327-server.virtnet: executing load_modules_local [ 7749.621930] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 7750.382143] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 7750.397273] Lustre: Skipped 3 previous similar messages [ 7755.329261] Lustre: DEBUG MARKER: oleg327-server.virtnet: executing set_default_debug -1 all [ 7763.921114] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 7768.128931] Lustre: DEBUG MARKER: oleg327-server.virtnet: executing set_default_debug -1 all [ 7770.740477] Lustre: 252890:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify general/debug_raw_pointers=Y (mode = 0) failed: rc = -17 [ 7777.349039] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 7783.772764] Lustre: DEBUG MARKER: oleg327-server.virtnet: executing set_default_debug -1 all [ 7790.015503] Lustre: lustre-OST0000: new connection from lustre-MDT0001-mdtlov (cleaning up unused objects from 0x280000400:99 to 0x280000400:193) [ 7793.223943] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 7795.664237] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:49698 to 0x280000401:49824) [ 7795.668616] Lustre: lustre-OST0001: new connection from lustre-MDT0001-mdtlov (cleaning up unused objects from 0x2c0000400:35 to 0x2c0000400:97) [ 7800.117333] Lustre: DEBUG MARKER: oleg327-server.virtnet: executing set_default_debug -1 all [ 7807.688867] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 7811.298870] Lustre: 254763:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify general/lod.*.mdt_hash=crush (mode = 0) failed: rc = -17 [ 7818.863374] Lustre: DEBUG MARKER: == conf-sanity test 104a: Make sure user defined options are reflected in mount ========================================================== 22:50:41 (1787799041) [ 7836.135498] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 7836.142830] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 7836.160570] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 7836.167879] Lustre: Skipped 4 previous similar messages [ 7839.888988] Lustre: server umount lustre-MDT0000 complete [ 7841.767925] LustreError: 251750:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 7841.782096] LustreError: 251750:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 7 previous similar messages [ 7844.136171] LustreError: 251726:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1787799068 with bad export cookie 14966718186513075477 [ 7844.146125] LustreError: MGC192.168.203.127@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 7844.152493] LustreError: 251726:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 3 previous similar messages [ 7846.898306] Lustre: lustre-MDT0001: Not available for connect from 0@lo (stopping) [ 7846.909491] Lustre: Skipped 4 previous similar messages [ 7850.775552] Lustre: server umount lustre-MDT0001 complete [ 7861.662034] Lustre: server umount lustre-OST0000 complete [ 7873.999663] Lustre: server umount lustre-OST0001 complete [ 7882.192705] Lustre: DEBUG MARKER: oleg327-server.virtnet: executing set_hostid [ 7892.825472] Lustre: DEBUG MARKER: oleg327-server.virtnet: executing load_modules_local [ 7927.566746] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 7927.888181] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 7927.939289] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 7928.103178] Lustre: lustre-MDT0000: new disk, initializing [ 7928.195983] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 188-180 [ 7928.204867] Lustre: Skipped 3 previous similar messages [ 7928.227555] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 7932.647338] Lustre: DEBUG MARKER: oleg327-server.virtnet: executing set_default_debug -1 all [ 7943.152289] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 7943.301841] Lustre: 257947:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 7943.384180] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [ 7943.388271] Lustre: Skipped 1 previous similar message [ 7943.478994] Lustre: lustre-MDT0001: new disk, initializing [ 7943.610938] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [ 7943.632766] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 7948.006069] Lustre: DEBUG MARKER: oleg327-server.virtnet: executing set_default_debug -1 all [ 7957.814910] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 7958.145837] Lustre: lustre-OST0000: new disk, initializing [ 7958.155752] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 7958.165719] Lustre: 259530:0:(osd_compat.c:1353:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 7959.874982] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [ 7959.883051] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 7959.967203] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 7964.339701] Lustre: DEBUG MARKER: oleg327-server.virtnet: executing set_default_debug -1 all [ 7975.080415] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 7975.336911] Lustre: lustre-OST0001: new disk, initializing [ 7975.345352] Lustre: srv-lustre-OST0001: No data found on store. Initialize space. [ 7975.360689] Lustre: 260400:0:(osd_compat.c:1353:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 7977.416649] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x00000002c0000400-0x0000000300000400]:1:ost [ 7977.435903] Lustre: cli-lustre-OST0001-super: Allocated super-sequence [0x00000002c0000400-0x0000000300000400]:1:ost] [ 7977.587800] Lustre: lustre-OST0001-osc-MDT0000: update sequence from 0x100010000 to 0x2c0000401 [ 7983.252288] Lustre: DEBUG MARKER: oleg327-server.virtnet: executing set_default_debug -1 all [ 7995.629139] Lustre: DEBUG MARKER: == conf-sanity test 104b: Mount uses last flock argument ========================================================== 22:53:38 (1787799218) [ 8008.547619] Lustre: DEBUG MARKER: == conf-sanity test 105: check file creation for ro and rw bind mnt pt ========================================================== 22:53:51 (1787799231) [ 8023.533777] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 8023.538801] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 8023.550436] Lustre: Skipped 6 previous similar messages [ 8023.568524] Lustre: Skipped 3 previous similar messages [ 8029.456291] Lustre: server umount lustre-OST0000 complete [ 8038.975605] Lustre: server umount lustre-MDT0000 complete [ 8042.579470] LustreError: 258742:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1787799266 with bad export cookie 14966718186513077255 [ 8042.585225] LustreError: MGC192.168.203.127@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 8042.590943] LustreError: 258742:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 2 previous similar messages [ 8048.949833] Lustre: server umount lustre-MDT0001 complete [ 8058.889729] Lustre: DEBUG MARKER: oleg327-server.virtnet: executing unload_modules_local [ 8129.165958] Lustre: server umount lustre-OST0001 complete [ 8136.742607] Lustre: DEBUG MARKER: oleg327-server.virtnet: executing set_hostid [ 8150.111984] Lustre: DEBUG MARKER: oleg327-server.virtnet: executing load_modules_local [ 8192.068869] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 8192.430307] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 8192.508147] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 8192.630224] Lustre: lustre-MDT0000: new disk, initializing [ 8192.764118] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 8192.767713] Lustre: Skipped 3 previous similar messages [ 8192.787252] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 8197.219335] Lustre: DEBUG MARKER: oleg327-server.virtnet: executing set_default_debug -1 all [ 8207.525973] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 8207.627433] Lustre: 265242:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 8207.917498] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 8212.746766] Lustre: DEBUG MARKER: oleg327-server.virtnet: executing set_default_debug -1 all [ 8222.142639] Lustre: DEBUG MARKER: oleg327-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 8227.173427] Lustre: DEBUG MARKER: oleg327-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 8233.567948] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 8233.778964] Lustre: lustre-OST0000: new disk, initializing [ 8233.783134] Lustre: Skipped 1 previous similar message [ 8233.786918] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 8233.793801] Lustre: Skipped 2 previous similar messages [ 8233.797940] Lustre: 267208:0:(osd_compat.c:1353:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 8235.365666] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [ 8235.382222] Lustre: Skipped 1 previous similar message [ 8235.386458] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 8235.461874] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 8239.353276] Lustre: DEBUG MARKER: oleg327-server.virtnet: executing set_default_debug -1 all [ 8248.430563] Lustre: DEBUG MARKER: oleg327-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 8255.984593] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 8255.985315] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 8255.997739] LustreError: Skipped 1 previous similar message [ 8256.013607] Lustre: Skipped 4 previous similar messages [ 8256.026575] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 8256.040120] Lustre: Skipped 11 previous similar messages [ 8260.839874] Lustre: server umount lustre-OST0000 complete [ 8270.750651] Lustre: server umount lustre-MDT0000 complete [ 8271.334066] LustreError: 265252:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 8271.344191] LustreError: 265252:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 9 previous similar messages [ 8274.388252] LustreError: 265234:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1787799498 with bad export cookie 14966718186513078725 [ 8274.390275] LustreError: MGC192.168.203.127@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 8274.415578] LustreError: 265234:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 8274.704205] Lustre: server umount lustre-MDT0001 complete [ 8286.461526] Lustre: DEBUG MARKER: oleg327-server.virtnet: executing unload_modules_local [ 8289.234710] Key type lgssc unregistered [ 8289.546420] LNet: 268973:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 8289.565736] LNetError: 268973:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 8290.598638] LNet: Removed LNI 192.168.203.127@tcp [ 8291.661338] Key type .llcrypt unregistered [ 8291.664364] Key type ._llcrypt unregistered [ 8315.811497] Key type ._llcrypt registered [ 8315.813257] Key type .llcrypt registered [ 8315.983867] Lustre: DEBUG MARKER: oleg327-server.virtnet: executing load_modules_local [ 8317.071821] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 8317.087450] alg: No test for adler32 (adler32-zlib) [ 8318.088454] Lustre: Lustre: Build Version: 2.17.57_50_g504224c [ 8318.304472] LNet: Added LNI 192.168.203.127@tcp [8/256/0/180] [ 8319.936187] Key type lgssc registered [ 8321.016289] Lustre: Echo OBD driver; http://www.lustre.org/ [ 8328.153484] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 8328.176496] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 8329.567753] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 8333.716358] Lustre: DEBUG MARKER: oleg327-server.virtnet: executing set_default_debug -1 all [ 8339.942422] LustreError: 270429:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 8339.955403] LustreError: 270429:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 2 previous similar messages [ 8341.625421] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 8341.850806] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 8346.629994] Lustre: DEBUG MARKER: oleg327-server.virtnet: executing set_default_debug -1 all [ 8354.673777] Lustre: DEBUG MARKER: oleg327-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 8360.908722] Lustre: DEBUG MARKER: oleg327-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 8367.865884] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 8368.200705] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 8374.272530] Lustre: DEBUG MARKER: oleg327-server.virtnet: executing set_default_debug -1 all [ 8374.391171] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:3 to 0x280000401:33) [ 8382.150672] Lustre: DEBUG MARKER: oleg327-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 8382.975892] Lustre: Setting parameter lustre-OST0000.ost.unknown_param=50 in log lustre-OST0000 [ 8384.996633] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 8384.999738] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 8385.037560] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 8386.089677] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 8391.012803] Lustre: server umount lustre-OST0000 complete [ 8396.258758] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 8396.270134] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 8396.273273] Lustre: Skipped 1 previous similar message [ 8401.274385] Lustre: server umount lustre-MDT0000 complete [ 8401.377562] LustreError: 271156:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 8401.397567] LustreError: 271156:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 1 previous similar message [ 8404.628438] LustreError: 270410:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1787799628 with bad export cookie 6297556640312329871 [ 8404.630545] LustreError: MGC192.168.203.127@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 8404.642081] LustreError: 270410:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 1 previous similar message [ 8405.037065] Lustre: server umount lustre-MDT0001 complete [ 8416.168492] Lustre: DEBUG MARKER: oleg327-server.virtnet: executing unload_modules_local [ 8419.204292] Key type lgssc unregistered [ 8419.510672] LNet: 274014:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 8419.533099] LNetError: 274014:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 8419.549784] LNet: Removed LNI 192.168.203.127@tcp [ 8420.451387] Key type .llcrypt unregistered [ 8420.455388] Key type ._llcrypt unregistered [ 8436.446703] Key type ._llcrypt registered [ 8436.450447] Key type .llcrypt registered [ 8436.601096] Lustre: DEBUG MARKER: oleg327-server.virtnet: executing load_modules_local [ 8438.266371] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 8438.296814] alg: No test for adler32 (adler32-zlib) [ 8439.341308] Lustre: Lustre: Build Version: 2.17.57_50_g504224c [ 8439.550672] LNet: Added LNI 192.168.203.127@tcp [8/256/0/180] [ 8441.184187] Key type lgssc registered [ 8442.313437] Lustre: Echo OBD driver; http://www.lustre.org/ [ 8450.659714] Lustre: lustre-OST0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 8450.677825] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 8466.208388] LustreError: Failed to get MGS log lustre-sptlrpc and no local copy. [ 8471.333232] LustreError: 275217:0:(mgc_request.c:233:do_config_log_add()) MGC192.168.203.127@tcp: failed processing log, type 4: rc = -110 [ 8497.952417] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 8505.807796] Lustre: DEBUG MARKER: oleg327-server.virtnet: executing set_default_debug -1 all [ 8513.510650] Lustre: DEBUG MARKER: oleg327-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 8515.744499] Lustre: server umount lustre-OST0000 complete [ 8532.035604] Lustre: DEBUG MARKER: oleg327-server.virtnet: executing unload_modules_local [ 8534.860583] Key type lgssc unregistered [ 8535.131754] LNet: 276851:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 8535.145198] LNetError: 276851:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 8535.159239] LNet: Removed LNI 192.168.203.127@tcp [ 8535.845684] Key type .llcrypt unregistered [ 8535.848274] Key type ._llcrypt unregistered [ 8568.642723] Key type ._llcrypt registered [ 8568.649365] Key type .llcrypt registered [ 8568.784252] Lustre: DEBUG MARKER: oleg327-server.virtnet: executing load_modules_local [ 8569.678989] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 8569.774506] alg: No test for adler32 (adler32-zlib) [ 8570.833843] Lustre: Lustre: Build Version: 2.17.57_50_g504224c [ 8571.049763] LNet: Added LNI 192.168.203.127@tcp [8/256/0/180] [ 8572.728211] Key type lgssc registered [ 8574.094893] Lustre: Echo OBD driver; http://www.lustre.org/ [ 8587.891990] loop0: detected capacity change from 0 to 204800000 [ 8588.058217] blk_update_request: operation not supported error, dev loop0, sector 399872 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [ 8588.079632] blk_update_request: operation not supported error, dev loop0, sector 208 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [ 8588.347258] blk_update_request: operation not supported error, dev loop0, sector 160232 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [ 8589.254620] loop0: detected capacity change from 0 to 204800000 [ 8589.387727] blk_update_request: operation not supported error, dev loop0, sector 399872 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [ 8589.401889] blk_update_request: operation not supported error, dev loop0, sector 208 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [ 8589.615940] blk_update_request: operation not supported error, dev loop0, sector 160232 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [ 8590.479962] loop0: detected capacity change from 0 to 204800000 [ 8590.703937] blk_update_request: operation not supported error, dev loop0, sector 399872 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [ 8590.720526] blk_update_request: operation not supported error, dev loop0, sector 8224 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [ 8590.850228] blk_update_request: operation not supported error, dev loop0, sector 58264 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [ 8591.927480] loop0: detected capacity change from 0 to 204800000 [ 8592.122669] blk_update_request: operation not supported error, dev loop0, sector 399872 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [ 8593.271880] loop0: detected capacity change from 0 to 204800000 [ 8593.304200] LDISKFS-fs (loop0): mounted filesystem with ordered data mode. Opts: (null) [ 8596.785567] loop0: detected capacity change from 0 to 204800000 [ 8596.818474] LDISKFS-fs (loop0): mounted filesystem with ordered data mode. Opts: (null) [ 8600.579601] loop0: detected capacity change from 0 to 204800000 [ 8600.609197] LDISKFS-fs (loop0): mounted filesystem with ordered data mode. Opts: (null) [ 8604.166264] loop0: detected capacity change from 0 to 204800000 [ 8604.186612] LDISKFS-fs (loop0): mounted filesystem with ordered data mode. Opts: (null) [ 8607.739821] loop0: detected capacity change from 0 to 204800000 [ 8607.848322] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 8607.883411] LDISKFS-fs (loop0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 8607.949736] Lustre: 280265:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0000: restore index 'fld' with [0x200000001:0x3:0x0]: rc = 0 [ 8607.956697] Lustre: 280265:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0000: restore index 'nodemap' with [0x200000003:0x3:0x0]: rc = 0 [ 8607.964460] Lustre: 280265:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0000: restore index '0x2010000' with [0x200000003:0xc:0x0]: rc = 0 [ 8607.971494] Lustre: 280265:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0000: restore index '0x1010000' with [0x200000003:0xb:0x0]: rc = 0 [ 8607.982604] Lustre: 280265:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0000: restore index '0x1010000-MDT0000' with [0x200000005:0x2:0x0]: rc = 0 [ 8607.992429] Lustre: 280265:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0000: restore index '0x10000' with [0x200000003:0xa:0x0]: rc = 0 [ 8608.006689] Lustre: 280265:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0000: restore index '0x10000-MDT0000' with [0x200000005:0x1:0x0]: rc = 0 [ 8608.019119] Lustre: 280265:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0000: restore index '0x2010000-MDT0000' with [0x200000005:0x3:0x0]: rc = 0 [ 8608.032259] Lustre: 280265:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0000: restore index '0x1020000' with [0x200000006:0x1020000:0x0]: rc = 0 [ 8608.041581] Lustre: 280265:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0000: restore index '0x20000' with [0x200000006:0x20000:0x0]: rc = 0 [ 8608.052100] Lustre: 280265:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0000: restore index '0x2020000' with [0x200000006:0x2020000:0x0]: rc = 0 [ 8608.061848] Lustre: 280265:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0000: restore index '0x2010000' with [0x200000006:0x2010000:0x0]: rc = 0 [ 8608.072512] Lustre: 280265:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0000: restore index '0x1010000' with [0x200000006:0x1010000:0x0]: rc = 0 [ 8608.091742] Lustre: 280265:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0000: restore index '0x10000' with [0x200000006:0x10000:0x0]: rc = 0 [ 8612.749661] Lustre: server umount MGS complete [ 8613.609074] loop0: detected capacity change from 0 to 204800000 [ 8613.693885] LDISKFS-fs (loop0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 8614.181617] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 8614.253671] Lustre: lustre-MDT0000: Aborting client recovery [ 8615.052304] loop1: detected capacity change from 0 to 204800000 [ 8615.229769] LDISKFS-fs (loop1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 8615.335658] Lustre: 280699:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0001: restore index 'fld' with [0x200000001:0x3:0x0]: rc = 0 [ 8615.344607] Lustre: 280699:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0001: restore index '0x10000' with [0x200000003:0x4:0x0]: rc = 0 [ 8615.359267] Lustre: 280699:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0001: restore index '0x2010000' with [0x200000003:0x6:0x0]: rc = 0 [ 8615.374944] Lustre: 280699:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0001: restore index '0x1010000-MDT0001' with [0x200000005:0x2:0x0]: rc = 0 [ 8615.391207] Lustre: 280699:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0001: restore index '0x2010000-MDT0001' with [0x200000005:0x3:0x0]: rc = 0 [ 8615.404248] Lustre: 280699:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0001: restore index '0x1010000' with [0x200000003:0x5:0x0]: rc = 0 [ 8615.413976] Lustre: 280699:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0001: restore index '0x10000-MDT0001' with [0x200000005:0x1:0x0]: rc = 0 [ 8615.439844] Lustre: Found index 1 for lustre-MDT0001, updating log [ 8615.703300] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 8615.772695] Lustre: lustre-MDT0001: Aborting client recovery [ 8616.679207] loop2: detected capacity change from 0 to 204800000 [ 8616.767794] LDISKFS-fs (loop2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 8616.822039] Lustre: 280779:0:(osd_scrub.c:1854:osd_index_restore()) lustre-OST0000: restore index 'fld' with [0x200000001:0x3:0x0]: rc = 0 [ 8616.832037] Lustre: 280779:0:(osd_scrub.c:1854:osd_index_restore()) lustre-OST0000: restore index '0x20000' with [0x200000003:0x5:0x0]: rc = 0 [ 8616.850057] Lustre: 280779:0:(osd_scrub.c:1854:osd_index_restore()) lustre-OST0000: restore index '0x20000-OST0000' with [0x200000005:0x1:0x0]: rc = 0 [ 8616.863786] Lustre: 280779:0:(osd_scrub.c:1854:osd_index_restore()) lustre-OST0000: restore index '0x2020000-OST0000' with [0x200000005:0x3:0x0]: rc = 0 [ 8616.880632] Lustre: 280779:0:(osd_scrub.c:1854:osd_index_restore()) lustre-OST0000: restore index '0x1020000-OST0000' with [0x200000005:0x2:0x0]: rc = 0 [ 8616.895525] Lustre: 280779:0:(osd_scrub.c:1854:osd_index_restore()) lustre-OST0000: restore index '0x2020000' with [0x200000003:0x7:0x0]: rc = 0 [ 8616.905568] Lustre: 280779:0:(osd_scrub.c:1854:osd_index_restore()) lustre-OST0000: restore index '0x1020000' with [0x200000003:0x6:0x0]: rc = 0 [ 8617.028665] Lustre: Found index 0 for lustre-OST0000, updating log [ 8617.079800] Lustre: 280796:0:(osd_compat.c:1353:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 8617.145493] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 8617.170596] Lustre: lustre-OST0000: Aborting recovery [ 8618.114700] loop3: detected capacity change from 0 to 204800000 [ 8618.261149] LDISKFS-fs (loop3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 8618.331039] Lustre: 280862:0:(osd_scrub.c:1854:osd_index_restore()) lustre-OST0001: restore index 'fld' with [0x200000001:0x3:0x0]: rc = 0 [ 8618.345574] Lustre: 280862:0:(osd_scrub.c:1854:osd_index_restore()) lustre-OST0001: restore index '0x20000' with [0x200000003:0x5:0x0]: rc = 0 [ 8618.358810] Lustre: 280862:0:(osd_scrub.c:1854:osd_index_restore()) lustre-OST0001: restore index '0x1020000-OST0001' with [0x200000005:0x2:0x0]: rc = 0 [ 8618.370829] Lustre: 280862:0:(osd_scrub.c:1854:osd_index_restore()) lustre-OST0001: restore index '0x2020000-OST0001' with [0x200000005:0x3:0x0]: rc = 0 [ 8618.378433] Lustre: 280862:0:(osd_scrub.c:1854:osd_index_restore()) lustre-OST0001: restore index '0x1020000' with [0x200000003:0x6:0x0]: rc = 0 [ 8618.387570] Lustre: 280862:0:(osd_scrub.c:1854:osd_index_restore()) lustre-OST0001: restore index '0x20000-OST0001' with [0x200000005:0x1:0x0]: rc = 0 [ 8618.395237] Lustre: 280862:0:(osd_scrub.c:1854:osd_index_restore()) lustre-OST0001: restore index '0x2020000' with [0x200000003:0x7:0x0]: rc = 0 [ 8618.407826] Lustre: Found index 1 for lustre-OST0001, updating log [ 8618.430384] Lustre: 280867:0:(osd_compat.c:1353:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 8618.540651] Lustre: lustre-OST0001: Aborting recovery [ 8623.626855] LustreError: 280643:0:(lod_dev.c:511:lod_sub_recovery_thread()) lustre-MDT0001-osp-MDT0000: get update log duration 9, retries 0, failed: rc = -108 [ 8624.615164] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x0:6 to 0x0:33) [ 8624.617632] Lustre: lustre-OST0001: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x0:6 to 0x0:33) [ 8624.647408] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000bd0 [ 8624.666455] Lustre: lustre-OST0001-osc-MDT0000: update sequence from 0x100010000 to 0x2c0000bd0 [ 8625.642769] Lustre: lustre-OST0000: new connection from lustre-MDT0001-mdtlov (cleaning up unused objects from 0x280000400:5 to 0x280000400:33) [ 8625.645341] Lustre: lustre-OST0001: new connection from lustre-MDT0001-mdtlov (cleaning up unused objects from 0x2c0000400:5 to 0x2c0000400:33) [ 8627.092192] LustreError: 280610:0:(mdt_identity.c:115:mdt_identity_do_upcall()) lustre-MDT0000: error invoking upcall /root/Work/Lustre/R5/lustre-release/lustre/tests/../utils/l_getidentity lustre-MDT0000 0: rc -2; check /proc/fs/lustre/mdt/lustre-MDT0000/identity_upcall, time 1427us: rc = -2 [ 8627.645933] LustreError: 280611:0:(mdt_identity.c:115:mdt_identity_do_upcall()) lustre-MDT0000: error invoking upcall /root/Work/Lustre/R5/lustre-release/lustre/tests/../utils/l_getidentity lustre-MDT0000 0: rc -2; check /proc/fs/lustre/mdt/lustre-MDT0000/identity_upcall, time 641us: rc = -2 [ 8627.667524] LustreError: 280611:0:(mdt_identity.c:115:mdt_identity_do_upcall()) Skipped 22 previous similar messages [ 8640.993112] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 8641.004337] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 8641.008489] Lustre: Skipped 1 previous similar message [ 8641.012953] Lustre: Skipped 2 previous similar messages [ 8646.114613] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 8646.120818] Lustre: Skipped 4 previous similar messages [ 8651.235429] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 8651.240609] Lustre: Skipped 3 previous similar messages [ 8654.306447] Lustre: lustre-MDT0000 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 8654.546959] Lustre: server umount lustre-MDT0000 complete [ 8655.301527] LustreError: 280594:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1787799879 with bad export cookie 6364357403792545715 [ 8655.312855] LustreError: MGC192.168.203.127@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 8656.356456] LustreError: 280608:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 8656.357074] Lustre: lustre-MDT0001-lwp-OST0000: Connection to lustre-MDT0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 8656.362263] Lustre: lustre-MDT0001: Not available for connect from 0@lo (stopping) [ 8656.379869] LustreError: 280608:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 3 previous similar messages [ 8656.403475] Lustre: Skipped 1 previous similar message [ 8660.389023] LustreError: 280806:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 8660.393925] Lustre: lustre-MDT0001: Not available for connect from 0@lo (stopping) [ 8660.415520] LustreError: 280806:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 1 previous similar message [ 8660.420654] Lustre: Skipped 2 previous similar messages [ 8661.720151] Lustre: server umount lustre-MDT0001 complete [ 8668.675080] Lustre: server umount lustre-OST0000 complete [ 8675.737335] Lustre: server umount lustre-OST0001 complete [ 8684.016380] Lustre: DEBUG MARKER: == conf-sanity test 109a: test lctl clear_conf fsname ==== 23:05:07 (1787799907) [ 8711.307934] Lustre: DEBUG MARKER: oleg327-server.virtnet: executing set_hostid [ 8722.051587] Lustre: DEBUG MARKER: oleg327-server.virtnet: executing load_modules_local [ 8757.219139] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 8757.472672] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 8757.531775] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 8757.609775] Lustre: lustre-MDT0000: new disk, initializing [ 8757.703502] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 8757.706696] Lustre: Skipped 1 previous similar message [ 8757.814699] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 8762.378862] Lustre: DEBUG MARKER: oleg327-server.virtnet: executing set_default_debug -1 all [ 8772.341632] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 8772.447637] Lustre: 284936:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 8772.504351] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [ 8772.508324] Lustre: Skipped 1 previous similar message [ 8772.575226] Lustre: lustre-MDT0001: new disk, initializing [ 8772.662247] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 8772.691105] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [ 8772.712587] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 8776.578646] Lustre: DEBUG MARKER: oleg327-server.virtnet: executing set_default_debug -1 all [ 8786.330613] Lustre: DEBUG MARKER: oleg327-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 8792.194983] Lustre: DEBUG MARKER: oleg327-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 8799.720621] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 8800.048176] Lustre: lustre-OST0000: new disk, initializing [ 8800.053370] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 8800.062548] Lustre: 286904:0:(osd_compat.c:1353:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 8800.170478] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 8806.626288] Lustre: DEBUG MARKER: oleg327-server.virtnet: executing set_default_debug -1 all [ 8807.521338] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [ 8807.533593] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 8807.683543] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 8817.370582] Lustre: DEBUG MARKER: oleg327-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 8823.320549] Lustre: Setting parameter lustre-MDT0000.mdd.atime_diff=62 in log lustre-MDT0000 [ 8835.615147] Lustre: Modifying parameter lustre-MDT0000.mdd.atime_diff=63 in log lustre-MDT0000 [ 8843.658917] Lustre: Setting parameter lustre-client.llite.max_read_ahead_mb=32 in log lustre-client [ 8905.196638] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 8905.197701] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 8905.201848] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 8905.217104] LustreError: Skipped 1 previous similar message [ 8905.252480] Lustre: Skipped 1 previous similar message [ 8909.466477] Lustre: server umount lustre-OST0000 complete [ 8915.427974] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 8915.462129] Lustre: Skipped 1 previous similar message [ 8919.648737] Lustre: server umount lustre-MDT0000 complete [ 8920.546597] LustreError: 284941:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 8920.574983] LustreError: 284941:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 1 previous similar message [ 8924.494590] LustreError: 284927:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1787800148 with bad export cookie 6364357403792548417 [ 8924.514741] LustreError: MGC192.168.203.127@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 8924.523443] LustreError: 284927:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 8925.052756] Lustre: server umount lustre-MDT0001 complete [ 8933.950918] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 8938.969719] Lustre: DEBUG MARKER: oleg327-server.virtnet: executing set_default_debug -1 all [ 8944.439486] Lustre: server umount MGS complete [ 8957.002919] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 8957.941398] LustreError: 290819:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 8958.030678] LustreError: 290832:0:(qmt_pool.c:1639:qmt_pool_add_rem()) lustre-QMT0000: can't remove lustre-OST0000_UUID pool 'pool1': rc = -22 [ 8958.157336] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 8963.561845] LustreError: 290820:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 8963.587846] LustreError: 290820:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 1 previous similar message [ 8964.010803] Lustre: DEBUG MARKER: oleg327-server.virtnet: executing set_default_debug -1 all [ 8973.805330] LustreError: 290819:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 8973.829203] LustreError: 290819:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 1 previous similar message [ 8977.797055] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 8984.480141] Lustre: DEBUG MARKER: oleg327-server.virtnet: executing set_default_debug -1 all [ 8992.319161] Lustre: DEBUG MARKER: oleg327-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 8998.551557] Lustre: DEBUG MARKER: oleg327-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 9006.061660] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 9006.397248] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 9006.403771] Lustre: Skipped 1 previous similar message [ 9013.843470] Lustre: DEBUG MARKER: oleg327-server.virtnet: executing set_default_debug -1 all [ 9022.889915] Lustre: DEBUG MARKER: oleg327-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 9065.954883] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 9065.960667] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 9065.985688] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 9065.990402] Lustre: Skipped 3 previous similar messages [ 9079.267930] Lustre: lustre-OST0000 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 9079.475321] Lustre: server umount lustre-OST0000 complete [ 9085.921821] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 9085.943123] LustreError: Skipped 1 previous similar message [ 9085.947756] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 9085.961465] Lustre: Skipped 1 previous similar message [ 9089.177736] Lustre: server umount lustre-MDT0000 complete [ 9091.554319] LustreError: 290814:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 9091.572321] LustreError: 290814:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 1 previous similar message [ 9093.285995] LustreError: 290801:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1787800317 with bad export cookie 6364357403792550265 [ 9093.291037] LustreError: MGC192.168.203.127@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 9093.303178] LustreError: 290801:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 9093.684417] Lustre: server umount lustre-MDT0001 complete [ 9108.277233] Lustre: DEBUG MARKER: oleg327-server.virtnet: executing unload_modules_local [ 9111.059176] Key type lgssc unregistered [ 9111.349659] LNet: 294796:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 9111.360645] LNetError: 294796:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 9111.386849] LNet: Removed LNI 192.168.203.127@tcp [ 9112.140983] Key type .llcrypt unregistered [ 9112.144060] Key type ._llcrypt unregistered [ 9151.191947] Key type ._llcrypt registered [ 9151.197907] Key type .llcrypt registered [ 9151.365808] Lustre: DEBUG MARKER: oleg327-server.virtnet: executing set_hostid [ 9168.571677] Lustre: DEBUG MARKER: oleg327-server.virtnet: executing load_modules_local [ 9169.653107] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 9169.704728] alg: No test for adler32 (adler32-zlib) [ 9170.849877] Lustre: Lustre: Build Version: 2.17.57_50_g504224c [ 9171.091607] LNet: Added LNI 192.168.203.127@tcp [8/256/0/180] [ 9172.776738] Key type lgssc registered [ 9174.099645] Lustre: Echo OBD driver; http://www.lustre.org/ [ 9212.270458] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 9212.321322] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 9213.568758] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 9213.605240] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 9213.700927] Lustre: lustre-MDT0000: new disk, initializing [ 9213.815941] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 9213.841959] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 9218.148620] Lustre: DEBUG MARKER: oleg327-server.virtnet: executing set_default_debug -1 all [ 9229.098904] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 9229.231835] Lustre: 298760:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 9229.343719] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [ 9229.348645] Lustre: Skipped 1 previous similar message [ 9229.435721] Lustre: lustre-MDT0001: new disk, initializing [ 9229.535204] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 9229.579865] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [ 9229.598189] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 9234.073702] Lustre: DEBUG MARKER: oleg327-server.virtnet: executing set_default_debug -1 all [ 9244.103865] Lustre: DEBUG MARKER: oleg327-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 9249.997334] Lustre: DEBUG MARKER: oleg327-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 9257.100503] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 9257.412422] Lustre: lustre-OST0000: new disk, initializing [ 9257.419068] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 9257.429496] Lustre: 300729:0:(osd_compat.c:1353:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 9257.550062] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 9264.498346] Lustre: DEBUG MARKER: oleg327-server.virtnet: executing set_default_debug -1 all [ 9267.799226] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [ 9267.820558] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 9267.938456] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 9274.134653] Lustre: DEBUG MARKER: oleg327-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 9282.019596] Lustre: Setting parameter lustre-MDT0000.mdd.atime_diff=62 in log lustre-MDT0000 [ 9294.529700] Lustre: Modifying parameter lustre-MDT0000.mdd.atime_diff=63 in log lustre-MDT0000 [ 9300.387870] Lustre: Setting parameter lustre-client.llite.max_read_ahead_mb=32 in log lustre-client [ 9355.243266] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 9355.244142] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 9355.276385] LustreError: Skipped 1 previous similar message [ 9355.298526] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 9355.308103] Lustre: Skipped 1 previous similar message [ 9359.640579] Lustre: server umount lustre-OST0000 complete [ 9365.483908] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 9365.492908] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 9365.514024] Lustre: Skipped 2 previous similar messages [ 9370.362494] Lustre: server umount lustre-MDT0000 complete [ 9374.644674] LustreError: 298754:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1787800598 with bad export cookie 1928096690334664313 [ 9374.650654] LustreError: MGC192.168.203.127@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 9374.657368] LustreError: 298754:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 9375.323246] Lustre: server umount lustre-MDT0001 complete [ 9386.329529] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 9392.917964] Lustre: DEBUG MARKER: oleg327-server.virtnet: executing set_default_debug -1 all [ 9399.180236] Lustre: server umount MGS complete [ 9412.539857] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 9413.524877] LustreError: 304599:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 9413.551588] LustreError: 304599:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 3 previous similar messages [ 9413.596401] LustreError: 304612:0:(qmt_pool.c:1639:qmt_pool_add_rem()) lustre-QMT0000: can't remove lustre-OST0000_UUID pool 'pool1': rc = -22 [ 9413.759926] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 9418.726504] LustreError: 304600:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 9419.608800] Lustre: DEBUG MARKER: oleg327-server.virtnet: executing set_default_debug -1 all [ 9423.841197] LustreError: 304599:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 9428.966740] LustreError: 304600:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 9431.182260] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 9431.702142] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 9437.085748] Lustre: DEBUG MARKER: oleg327-server.virtnet: executing set_default_debug -1 all [ 9445.040218] Lustre: DEBUG MARKER: oleg327-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 9451.077135] Lustre: DEBUG MARKER: oleg327-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 9457.932151] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 9458.295802] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 9464.825558] Lustre: DEBUG MARKER: oleg327-server.virtnet: executing set_default_debug -1 all [ 9472.322920] Lustre: DEBUG MARKER: oleg327-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 9502.693706] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 9502.696045] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 9502.708566] Lustre: Skipped 1 previous similar message [ 9502.709200] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 9502.731287] Lustre: Skipped 2 previous similar messages [ 9507.822326] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 9507.830735] Lustre: Skipped 1 previous similar message [ 9508.909107] Lustre: server umount lustre-OST0000 complete [ 9512.933455] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 9512.935584] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 9512.955603] Lustre: Skipped 1 previous similar message [ 9512.964203] Lustre: Skipped 1 previous similar message [ 9518.635176] Lustre: server umount lustre-MDT0000 complete [ 9522.667526] LustreError: 304580:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1787800746 with bad export cookie 1928096690334666154 [ 9522.674700] LustreError: MGC192.168.203.127@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 9522.691159] LustreError: 304580:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 1 previous similar message [ 9523.096520] Lustre: server umount lustre-MDT0001 complete [ 9534.081610] Lustre: DEBUG MARKER: oleg327-server.virtnet: executing unload_modules_local [ 9537.486705] Key type lgssc unregistered [ 9537.828854] LNet: 308550:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 9537.837288] LNetError: 308550:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 9537.862541] LNet: Removed LNI 192.168.203.127@tcp [ 9538.734524] Key type .llcrypt unregistered [ 9538.739743] Key type ._llcrypt unregistered [ 9560.799164] Key type ._llcrypt registered [ 9560.807455] Key type .llcrypt registered [ 9560.957304] Lustre: DEBUG MARKER: oleg327-server.virtnet: executing set_hostid [ 9574.994212] Lustre: DEBUG MARKER: oleg327-server.virtnet: executing load_modules_local [ 9576.303966] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 9576.371104] alg: No test for adler32 (adler32-zlib) [ 9577.583644] Lustre: Lustre: Build Version: 2.17.57_50_g504224c [ 9577.966979] LNet: Added LNI 192.168.203.127@tcp [8/256/0/180] [ 9579.648172] Key type lgssc registered [ 9580.643482] Lustre: Echo OBD driver; http://www.lustre.org/ [ 9605.536873] Lustre: DEBUG MARKER: oleg327-server.virtnet: executing load_modules_local [ 9620.014992] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 9620.041776] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 9621.260288] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 9621.316961] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 9621.372400] Lustre: lustre-MDT0000: new disk, initializing [ 9621.430555] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 9621.446058] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 9625.693964] Lustre: DEBUG MARKER: oleg327-server.virtnet: executing set_default_debug -1 all [ 9635.053527] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 9635.255644] Lustre: lustre-OST0000: new disk, initializing [ 9635.259461] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 9635.263341] Lustre: Skipped 1 previous similar message [ 9635.267680] Lustre: 312601:0:(osd_compat.c:1353:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 9635.347310] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 9637.475744] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:0:ost [ 9637.486461] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:0:ost] [ 9637.536450] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x240000400 [ 9641.487913] Lustre: DEBUG MARKER: oleg327-server.virtnet: executing set_default_debug -1 all [10512.867716] Lustre: lustre-MDT0000-lwp-OST0000: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [10512.888073] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [10515.882552] Lustre: server umount lustre-MDT0000 complete [10519.139141] LustreError: 311787:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1787801743 with bad export cookie 14003588452484617140 [10519.148538] LustreError: MGC192.168.203.127@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [10519.152247] LustreError: 311787:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 2 previous similar messages [10519.336639] Lustre: server umount lustre-OST0000 complete [10543.219162] Lustre: DEBUG MARKER: oleg327-server.virtnet: executing set_hostid [10553.520127] Lustre: DEBUG MARKER: oleg327-server.virtnet: executing load_modules_local [10592.922520] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [10593.191861] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [10593.240568] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [10593.355786] Lustre: lustre-MDT0000: new disk, initializing [10593.551127] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [10593.574623] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [10598.180640] Lustre: DEBUG MARKER: oleg327-server.virtnet: executing set_default_debug -1 all [10611.122644] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [10611.324846] Lustre: 316945:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [10611.379655] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [10611.383830] Lustre: Skipped 1 previous similar message [10611.505930] Lustre: lustre-MDT0001: new disk, initializing [10611.691712] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [10611.746279] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [10611.758518] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [10616.380388] Lustre: DEBUG MARKER: oleg327-server.virtnet: executing set_default_debug -1 all [10625.976839] Lustre: DEBUG MARKER: oleg327-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [10631.744325] Lustre: DEBUG MARKER: oleg327-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [10638.586233] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [10638.869731] Lustre: lustre-OST0000: new disk, initializing [10638.879879] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [10638.894109] Lustre: 318912:0:(osd_compat.c:1353:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [10639.015811] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [10646.179461] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [10646.186470] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [10646.259989] Lustre: DEBUG MARKER: oleg327-server.virtnet: executing set_default_debug -1 all [10646.295180] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [10656.428758] Lustre: DEBUG MARKER: oleg327-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [10661.470887] Lustre: DEBUG MARKER: oleg327-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 [10661.654344] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [10666.061442] Lustre: DEBUG MARKER: oleg327-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid 50 [10666.206894] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [10672.104588] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [10672.109474] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [10672.134448] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [10672.144193] LustreError: Skipped 1 previous similar message [10672.171129] Lustre: Skipped 1 previous similar message [10673.765439] Lustre: server umount lustre-OST0000 complete [10678.241751] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [10678.249127] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [10678.256484] Lustre: Skipped 1 previous similar message [10678.265291] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [10682.342185] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [10682.348445] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [10683.525279] Lustre: server umount lustre-MDT0000 complete [10686.818318] LustreError: 316937:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1787801910 with bad export cookie 14003588452487557966 [10686.819138] LustreError: MGC192.168.203.127@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [10686.834661] LustreError: 316937:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 1 previous similar message [10687.112677] Lustre: server umount lustre-MDT0001 complete [10690.429785] Lustre: DEBUG MARKER: SKIP: conf-sanity test_111 skipping SLOW test 111 [10692.341328] Lustre: DEBUG MARKER: == conf-sanity test 112a: mount OST with no_create option ========================================================== 23:38:35 (1787801915) [10700.036356] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [10700.617216] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [10705.691283] Lustre: DEBUG MARKER: oleg327-server.virtnet: executing set_default_debug -1 all [10705.897338] LustreError: 321224:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [10705.924512] LustreError: 321224:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 2 previous similar messages [10709.990471] LustreError: 321223:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [10715.107246] LustreError: 321224:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [10717.509351] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [10717.986057] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [10723.210431] Lustre: DEBUG MARKER: oleg327-server.virtnet: executing set_default_debug -1 all [10731.134386] Lustre: DEBUG MARKER: oleg327-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [10738.197309] Lustre: DEBUG MARKER: oleg327-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [10745.420698] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [10745.912075] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [10752.285207] Lustre: DEBUG MARKER: oleg327-server.virtnet: executing set_default_debug -1 all [10759.601355] Lustre: DEBUG MARKER: oleg327-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [10767.158681] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [10767.319785] Lustre: lustre-OST0001: new disk, initializing [10767.324177] Lustre: srv-lustre-OST0001: No data found on store. Initialize space. [10767.331429] Lustre: 324059:0:(osd_compat.c:1353:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [10767.420480] Lustre: lustre-OST0001: Imperative Recovery not enabled, recovery window 60-180 [10774.061804] Lustre: DEBUG MARKER: oleg327-server.virtnet: executing set_default_debug -1 all [10774.612321] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x00000002c0000400-0x0000000300000400]:1:ost [10774.623712] Lustre: cli-lustre-OST0001-super: Allocated super-sequence [0x00000002c0000400-0x0000000300000400]:1:ost] [10774.691194] LustreError: 324441:0:(osp_precreate.c:974:osp_precreate_cleanup_orphans()) lustre-OST0001-osc-MDT0001: cannot cleanup orphans: rc = -1 [10775.719853] LustreError: 324441:0:(osp_precreate.c:974:osp_precreate_cleanup_orphans()) lustre-OST0001-osc-MDT0001: cannot cleanup orphans: rc = -1 [10775.744027] LustreError: 324441:0:(osp_precreate.c:974:osp_precreate_cleanup_orphans()) Skipped 1 previous similar message [10776.808606] LustreError: 324446:0:(osp_precreate.c:974:osp_precreate_cleanup_orphans()) lustre-OST0001-osc-MDT0000: cannot cleanup orphans: rc = -1 [10776.827260] LustreError: 324446:0:(osp_precreate.c:974:osp_precreate_cleanup_orphans()) Skipped 1 previous similar message [10778.920438] LustreError: 324446:0:(osp_precreate.c:974:osp_precreate_cleanup_orphans()) lustre-OST0001-osc-MDT0000: cannot cleanup orphans: rc = -1 [10778.935286] LustreError: 324446:0:(osp_precreate.c:974:osp_precreate_cleanup_orphans()) Skipped 3 previous similar messages [10783.010645] LustreError: 324441:0:(osp_precreate.c:974:osp_precreate_cleanup_orphans()) lustre-OST0001-osc-MDT0001: cannot cleanup orphans: rc = -1 [10783.024087] LustreError: 324441:0:(osp_precreate.c:974:osp_precreate_cleanup_orphans()) Skipped 6 previous similar messages [10783.621836] Lustre: DEBUG MARKER: oleg327-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 [10783.823272] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [10789.622268] Lustre: DEBUG MARKER: oleg327-client.virtnet: executing wait_import_state (FULL|IDLE) osc.lustre-OST0000-osc-ffff9afec8fa1000.ost_server_uuid 50 [10791.100241] Lustre: DEBUG MARKER: osc.lustre-OST0000-osc-ffff9afec8fa1000.ost_server_uuid in FULL state after 0 sec [10791.269337] LustreError: 324441:0:(osp_precreate.c:974:osp_precreate_cleanup_orphans()) lustre-OST0001-osc-MDT0001: cannot cleanup orphans: rc = -1 [10791.282848] LustreError: 324441:0:(osp_precreate.c:974:osp_precreate_cleanup_orphans()) Skipped 15 previous similar messages [10796.891926] Lustre: DEBUG MARKER: oleg327-client.virtnet: executing wait_import_state (FULL|IDLE) osc.lustre-OST0001-osc-ffff9afec8fa1000.ost_server_uuid 50 [10798.517838] Lustre: DEBUG MARKER: osc.lustre-OST0001-osc-ffff9afec8fa1000.ost_server_uuid in FULL state after 0 sec [10800.589214] Lustre: lustre-OST0001-osc-MDT0000: update sequence from 0x100010000 to 0x2c0000401 [10815.268228] Lustre: lustre-OST0001: Not available for connect from 192.168.203.27@tcp (stopping) [10815.274452] Lustre: Skipped 1 previous similar message [10815.969904] LustreError: lustre-OST0001-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [10815.990685] Lustre: lustre-OST0001-osc-MDT0001: Connection to lustre-OST0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [10825.467275] Lustre: lustre-OST0001: Not available for connect from 192.168.203.27@tcp (stopping) [10825.470745] Lustre: Skipped 5 previous similar messages [10828.257064] Lustre: lustre-OST0001 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [10828.421795] Lustre: server umount lustre-OST0001 complete [10830.596036] LustreError: 323045:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-OST0001: not available for connect from 192.168.203.27@tcp (no target). If you are running an HA pair check that the target is mounted on the other server. [10836.456882] LustreError: 323043:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [10836.460126] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [10836.476513] LustreError: 323043:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 3 previous similar messages [10836.496362] Lustre: Skipped 1 previous similar message [10841.289405] Lustre: server umount lustre-OST0000 complete [10846.182838] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [10846.189077] LustreError: Skipped 1 previous similar message [10846.201655] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [10846.213933] Lustre: Skipped 4 previous similar messages [10852.140790] Lustre: server umount lustre-MDT0000 complete [10855.820246] LustreError: 321205:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1787802079 with bad export cookie 14003588452487558841 [10855.825663] LustreError: MGC192.168.203.127@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [10855.836863] LustreError: 321205:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [10856.187351] Lustre: server umount lustre-MDT0001 complete [10867.355969] Lustre: DEBUG MARKER: oleg327-server.virtnet: executing unload_modules_local [10870.209253] Key type lgssc unregistered [10870.543757] LNet: 326751:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [10870.555764] LNetError: 326751:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [10870.577342] LNet: Removed LNI 192.168.203.127@tcp [10871.418636] Key type .llcrypt unregistered [10871.423091] Key type ._llcrypt unregistered [10893.402868] Key type ._llcrypt registered [10893.406663] Key type .llcrypt registered [10893.504580] Lustre: DEBUG MARKER: oleg327-server.virtnet: executing load_modules_local [10894.672164] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [10894.784424] alg: No test for adler32 (adler32-zlib) [10895.908162] Lustre: Lustre: Build Version: 2.17.57_50_g504224c [10896.143217] LNet: Added LNI 192.168.203.127@tcp [8/256/0/180] [10897.784388] Key type lgssc registered [10898.657067] Lustre: Echo OBD driver; http://www.lustre.org/ [10906.433074] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [10906.454255] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [10907.920046] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [10912.037732] Lustre: DEBUG MARKER: oleg327-server.virtnet: executing set_default_debug -1 all [10917.354183] LustreError: 328172:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [10917.379465] LustreError: 328172:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 2 previous similar messages [10919.244586] Lustre: DEBUG MARKER: oleg327-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [10922.465576] LustreError: 328173:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [10926.563985] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [10926.938708] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [10931.573439] Lustre: DEBUG MARKER: oleg327-server.virtnet: executing set_default_debug -1 all [10942.399902] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [10942.728205] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [10948.662179] Lustre: DEBUG MARKER: oleg327-server.virtnet: executing set_default_debug -1 all [10951.988038] LustreError: 329854:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [10953.033246] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:36 to 0x280000401:65) [10955.736680] Lustre: DEBUG MARKER: oleg327-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [10958.309345] LustreError: 330363:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [10958.323503] LustreError: 330363:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 3 previous similar messages [10962.251031] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [10962.482103] Lustre: lustre-OST0001: Imperative Recovery not enabled, recovery window 60-180 [10967.560186] Lustre: lustre-OST0001: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x2c0000401:3 to 0x2c0000401:33) [10968.327288] Lustre: DEBUG MARKER: oleg327-server.virtnet: executing set_default_debug -1 all [10977.091742] Lustre: DEBUG MARKER: oleg327-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0001-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [10984.271157] Lustre: DEBUG MARKER: oleg327-server.virtnet: executing wait_import_state (FULL|IDLE) os[cp].lustre-OST0001-osc-MDT0001.ost_server_uuid 50 [10984.453827] Lustre: DEBUG MARKER: os[cp].lustre-OST0001-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [11095.526579] LustreError: lustre-OST0001-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [11095.529398] Lustre: lustre-OST0001-osc-MDT0001: Connection to lustre-OST0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [11095.534375] LustreError: Skipped 1 previous similar message [11095.536480] Lustre: lustre-OST0001: Not available for connect from 0@lo (stopping) [11095.561623] Lustre: Skipped 1 previous similar message [11098.331804] Lustre: server umount lustre-OST0001 complete [11100.652663] LustreError: 329855:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [11100.675727] LustreError: 329855:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 1 previous similar message [11105.761301] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [11105.773157] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [11105.777809] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [11105.791344] Lustre: Skipped 1 previous similar message [11105.796527] Lustre: Skipped 1 previous similar message [11110.884880] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [11110.886162] LustreError: 329855:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [11110.901826] Lustre: Skipped 2 previous similar messages [11110.933282] LustreError: 329855:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 3 previous similar messages [11111.350150] Lustre: server umount lustre-OST0000 complete [11116.514511] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [11116.523487] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [11116.543484] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [11119.098168] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [11121.123963] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [11121.135085] Lustre: Skipped 3 previous similar messages [11122.439442] Lustre: server umount lustre-MDT0000 complete [11126.230803] LustreError: 328153:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1787802350 with bad export cookie 7639159587011765267 [11126.233546] LustreError: MGC192.168.203.127@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [11126.252789] LustreError: 328153:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [11126.546101] Lustre: server umount lustre-MDT0001 complete [11139.294980] Lustre: DEBUG MARKER: oleg327-server.virtnet: executing unload_modules_local [11142.485368] Key type lgssc unregistered [11142.830244] LNet: 333122:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [11142.844847] LNetError: 333122:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [11142.868989] LNet: Removed LNI 192.168.203.127@tcp [11143.753835] Key type .llcrypt unregistered [11143.755957] Key type ._llcrypt unregistered [11195.190454] Key type ._llcrypt registered [11195.193829] Key type .llcrypt registered [11195.336547] Lustre: DEBUG MARKER: oleg327-server.virtnet: executing load_modules_local [11196.721711] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [11196.761904] alg: No test for adler32 (adler32-zlib) [11197.871337] Lustre: Lustre: Build Version: 2.17.57_50_g504224c [11198.086583] LNet: Added LNI 192.168.203.127@tcp [8/256/0/180] [11199.736224] Key type lgssc registered [11200.801163] Lustre: Echo OBD driver; http://www.lustre.org/ [11209.745949] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [11209.769839] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [11211.209593] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [11215.458552] Lustre: DEBUG MARKER: oleg327-server.virtnet: executing set_default_debug -1 all [11221.482269] LustreError: 335411:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [11221.503621] LustreError: 335411:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 2 previous similar messages [11223.381892] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [11223.766311] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [11228.239802] Lustre: DEBUG MARKER: oleg327-server.virtnet: executing set_default_debug -1 all [11231.300683] Lustre: Modifying parameter general.debug_raw_pointers=Y in log params [11237.791749] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [11238.124966] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [11243.845183] Lustre: DEBUG MARKER: oleg327-server.virtnet: executing set_default_debug -1 all [11248.381379] LustreError: 336903:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [11252.280787] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [11252.613451] Lustre: lustre-OST0001: Imperative Recovery not enabled, recovery window 60-180 [11256.698660] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:36 to 0x280000401:97) [11256.722824] Lustre: lustre-OST0001: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x2c0000401:3 to 0x2c0000401:65) [11259.338122] Lustre: DEBUG MARKER: oleg327-server.virtnet: executing set_default_debug -1 all [11268.914894] Lustre: DEBUG MARKER: Using TIMEOUT=20 [11278.174271] Lustre: Setting parameter general.lod.*.mdt_hash=crush in log params [11292.645059] Lustre: lustre-MDT0000-lwp-OST0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [11292.646869] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [11292.693655] Lustre: Skipped 3 previous similar messages [11292.711858] Lustre: Skipped 3 previous similar messages [11297.773440] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [11297.786846] Lustre: Skipped 3 previous similar messages [11298.213331] Lustre: server umount lustre-MDT0000 complete [11302.300945] LustreError: 335393:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1787802526 with bad export cookie 17548438519486065585 [11302.314644] LustreError: MGC192.168.203.127@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [11302.318972] LustreError: 335393:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [11302.711994] Lustre: server umount lustre-MDT0001 complete [11317.065457] Lustre: server umount lustre-OST0000 complete [11318.178514] Lustre: 334674:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1787802526/real 1787802526] req@ffff92b0bd7bf840 x1874646712515072/t0(0) o400->lustre-MDT0001-lwp-OST0001@0@lo:12/10 lens 224/224 e 0 to 1 dl 1787802542 ref 1 fl Rpc:XNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 [11318.226524] Lustre: lustre-MDT0001-lwp-OST0001: Connection to lustre-MDT0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [11321.388290] Lustre: server umount lustre-OST0001 complete [11343.348743] Lustre: DEBUG MARKER: oleg327-server.virtnet: executing load_modules_local [11353.427487] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [11353.816240] LustreError: 340681:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [11353.884482] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [11358.766698] Lustre: DEBUG MARKER: oleg327-server.virtnet: executing set_default_debug -1 all [11359.203069] LustreError: 340682:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [11363.313216] LustreError: 340681:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [11367.110512] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [11367.494701] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [11371.703743] Lustre: DEBUG MARKER: oleg327-server.virtnet: executing set_default_debug -1 all [11374.554924] Lustre: 341821:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify general/debug_raw_pointers=Y (mode = 0) failed: rc = -17 [11380.846756] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [11386.558656] Lustre: DEBUG MARKER: oleg327-server.virtnet: executing set_default_debug -1 all [11392.433644] LustreError: 342176:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [11394.874532] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [11395.151940] Lustre: lustre-OST0001: Imperative Recovery not enabled, recovery window 60-180 [11395.158468] Lustre: Skipped 1 previous similar message [11398.273396] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:36 to 0x280000401:129) [11398.295808] Lustre: lustre-OST0001: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x2c0000401:3 to 0x2c0000401:97) [11401.188974] Lustre: DEBUG MARKER: oleg327-server.virtnet: executing set_default_debug -1 all [11408.931728] Lustre: DEBUG MARKER: Using TIMEOUT=20 [11412.753234] Lustre: 343692:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify general/lod.*.mdt_hash=crush (mode = 0) failed: rc = -17 [11418.595660] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [11418.604843] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [11418.635513] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [11423.641907] Lustre: server umount lustre-MDT0000 complete [11424.227725] LustreError: 340676:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [11424.257989] LustreError: 340676:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 3 previous similar messages [11427.123600] LustreError: 340661:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1787802651 with bad export cookie 17548438519486067286 [11427.136839] LustreError: MGC192.168.203.127@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [11427.429429] Lustre: server umount lustre-MDT0001 complete [11441.731940] Lustre: server umount lustre-OST0000 complete [11455.892984] Lustre: server umount lustre-OST0001 complete [11459.719179] Lustre: DEBUG MARKER: SKIP: conf-sanity test_114 skipping SLOW test 114 [11461.514148] Lustre: DEBUG MARKER: SKIP: conf-sanity test_115 skipping excluded test 115 [11463.839469] Lustre: DEBUG MARKER: == conf-sanity test 116: big size MDT support ============ 23:51:26 (1787802686) [11489.252133] Lustre: DEBUG MARKER: oleg327-server.virtnet: executing load_modules_local [11498.796398] loop0: detected capacity change from 0 to 1099511627776 [11499.428494] SGI XFS with ACLs, security attributes, quota, no debug enabled [11499.435536] *** VALIDATE xfs *** [11499.444855] XFS (loop0): Mounting V5 Filesystem [11499.487451] XFS (loop0): Ending clean mount [11500.817715] loop1: detected capacity change from 0 to 18691697672192 [11610.610295] Lustre: DEBUG MARKER: oleg327-server.virtnet: executing set_hostid [11622.958832] Lustre: DEBUG MARKER: oleg327-server.virtnet: executing load_modules_local [11662.379724] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [11662.616907] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [11662.657182] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [11662.752917] Lustre: lustre-MDT0000: new disk, initializing [11662.857636] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [11662.879711] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [11667.608250] Lustre: DEBUG MARKER: oleg327-server.virtnet: executing set_default_debug -1 all [11677.276388] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [11677.392712] Lustre: 348832:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [11677.438747] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [11677.442683] Lustre: Skipped 1 previous similar message [11677.496962] Lustre: lustre-MDT0001: new disk, initializing [11677.613459] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [11677.632237] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [11681.991155] Lustre: DEBUG MARKER: oleg327-server.virtnet: executing set_default_debug -1 all [11691.139692] Lustre: DEBUG MARKER: oleg327-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [11697.129371] Lustre: DEBUG MARKER: oleg327-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [11703.759536] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [11704.122871] Lustre: lustre-OST0000: new disk, initializing [11704.133557] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [11704.154843] Lustre: 350802:0:(osd_compat.c:1353:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [11705.435514] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [11705.451632] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [11705.558633] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [11710.882801] Lustre: DEBUG MARKER: oleg327-server.virtnet: executing set_default_debug -1 all [11721.829284] Lustre: DEBUG MARKER: oleg327-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [11728.142930] Lustre: DEBUG MARKER: oleg327-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 [11728.455771] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [11734.487188] Lustre: DEBUG MARKER: oleg327-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid 50 [11734.709748] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [11741.667802] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [11741.669361] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [11741.676740] LustreError: Skipped 1 previous similar message [11741.679976] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [11741.684149] Lustre: Skipped 4 previous similar messages [11741.702240] Lustre: Skipped 5 previous similar messages [11746.792894] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [11746.810243] Lustre: Skipped 1 previous similar message [11751.392860] Lustre: lustre-OST0000 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [11751.869971] Lustre: server umount lustre-OST0000 complete [11756.158876] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [11756.166715] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [11756.188299] Lustre: Skipped 1 previous similar message [11756.209899] Lustre: Skipped 1 previous similar message [11762.596436] Lustre: server umount lustre-MDT0000 complete [11766.764682] LustreError: 348841:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [11766.801747] LustreError: 348841:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 1 previous similar message [11767.159256] LustreError: 348825:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1787802991 with bad export cookie 17548438519486068833 [11767.166356] LustreError: MGC192.168.203.127@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [11767.179176] LustreError: 348825:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 8 previous similar messages [11767.598134] Lustre: server umount lustre-MDT0001 complete [11777.249658] XFS (loop0): Unmounting Filesystem [11782.648640] Lustre: DEBUG MARKER: == conf-sanity test 117: lctl get_param return errors properly ========================================================== 23:56:45 (1787803005) [11792.237475] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [11793.233439] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [11793.244831] Lustre: Skipped 2 previous similar messages [11799.156600] Lustre: DEBUG MARKER: oleg327-server.virtnet: executing set_default_debug -1 all [11808.972025] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [11814.280784] Lustre: DEBUG MARKER: oleg327-server.virtnet: executing set_default_debug -1 all [11823.600529] Lustre: DEBUG MARKER: oleg327-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [11830.687134] Lustre: DEBUG MARKER: oleg327-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [11837.717060] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [11844.588095] Lustre: DEBUG MARKER: oleg327-server.virtnet: executing set_default_debug -1 all [11852.459545] Lustre: DEBUG MARKER: oleg327-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [11865.059842] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [11865.071861] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [11865.081494] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [11865.081502] Lustre: Skipped 2 previous similar messages [11865.085177] LustreError: Skipped 1 previous similar message [11865.110119] Lustre: Skipped 1 previous similar message [11870.407376] Lustre: server umount lustre-OST0000 complete [11880.632539] Lustre: server umount lustre-MDT0000 complete [11884.286432] LustreError: 353345:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1787803108 with bad export cookie 17548438519486069722 [11884.290930] LustreError: MGC192.168.203.127@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [11884.304233] LustreError: 353345:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 1 previous similar message [11884.672193] Lustre: server umount lustre-MDT0001 complete [11895.440734] Lustre: DEBUG MARKER: oleg327-server.virtnet: executing unload_modules_local [11899.039666] Key type lgssc unregistered [11899.523868] LNet: 356997:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [11899.537711] LNetError: 356997:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [11899.570211] LNet: Removed LNI 192.168.203.127@tcp [11900.543316] Key type .llcrypt unregistered [11900.549256] Key type ._llcrypt unregistered [11934.829901] Key type ._llcrypt registered [11934.833618] Key type .llcrypt registered [11934.957875] Lustre: DEBUG MARKER: oleg327-server.virtnet: executing load_modules_local [11936.391484] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [11936.432662] alg: No test for adler32 (adler32-zlib) [11937.471156] Lustre: Lustre: Build Version: 2.17.57_50_g504224c [11937.737058] LNet: Added LNI 192.168.203.127@tcp [8/256/0/180] [11939.424254] Key type lgssc registered [11940.746357] Lustre: Echo OBD driver; http://www.lustre.org/ [11948.638711] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [11948.668412] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [11950.176308] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [11954.105339] Lustre: DEBUG MARKER: oleg327-server.virtnet: executing set_default_debug -1 all [11960.810461] LustreError: 358566:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [11960.826090] LustreError: 358566:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 2 previous similar messages [11962.001354] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [11962.389825] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [11966.952151] Lustre: DEBUG MARKER: oleg327-server.virtnet: executing set_default_debug -1 all [11976.481840] Lustre: DEBUG MARKER: oleg327-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [11982.203748] Lustre: DEBUG MARKER: oleg327-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [11989.235138] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [11989.668352] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [11995.767911] Lustre: DEBUG MARKER: oleg327-server.virtnet: executing set_default_debug -1 all [12005.715665] Lustre: DEBUG MARKER: oleg327-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [12013.543098] LustreError: lustre-MDT0001-osp-MDT0000: operation mds_statfs to node 0@lo failed: rc = -107 [12013.548096] Lustre: lustre-MDT0001-osp-MDT0000: Connection to lustre-MDT0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [12013.564659] Lustre: lustre-MDT0001: Not available for connect from 0@lo (stopping) [12017.648934] Lustre: lustre-MDT0001: Not available for connect from 0@lo (stopping) [12017.672088] Lustre: lustre-MDT0001-lwp-OST0000: Connection to lustre-MDT0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [12019.724092] Lustre: server umount lustre-MDT0001 complete [12022.764460] LustreError: 358562:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [12022.801126] LustreError: 358562:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 1 previous similar message [12027.886714] LustreError: 359576:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [12027.917268] LustreError: 359576:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 2 previous similar messages [12031.303710] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [12031.462428] Lustre: MGS: Regenerating lustre-MDT0001 log by user request: rc = 0 [12031.471522] Lustre: Found index 1 for lustre-MDT0001, updating log [12031.524336] Lustre: 358555:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [12032.043621] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [12037.125591] Lustre: lustre-MDT0001-lwp-OST0000: Connection restored to 0@lo (at 0@lo) [12037.830277] Lustre: DEBUG MARKER: oleg327-server.virtnet: executing set_default_debug -1 all [12039.710140] LustreError: 362109:0:(obd_mount.c:234:lustre_start_simple()) lustre-MDT0001-lwp-OST0000 attach error -17 [12039.724059] LustreError: 362109:0:(tgt_mount.c:675:lustre_lwp_setup()) lustre-MDT0001-lwp-OST0000: setup up failed: rc -17 [12039.745266] LustreError: 358557:0:(mgc_request.c:614:do_requeue()) failed processing log: -17 [12039.823781] LustreError: 362117:0:(obd_config.c:2102:class_config_llog_handler()) MGC192.168.203.127@tcp: cfg command failed: rc = -17 [12039.835467] Lustre: cmd=cf001 0:lustre-MDT0001-osp-MDT0000 1:osp 2:lustre-MDT0000-mdtlov_UUID [12039.835467] [12048.677952] Lustre: lustre-MDT0001: Not available for connect from 192.168.203.27@tcp (stopping) [12048.693563] Lustre: Skipped 2 previous similar messages [12052.454717] LustreError: lustre-MDT0001-osp-MDT0000: operation mds_statfs to node 0@lo failed: rc = -107 [12052.469917] Lustre: lustre-MDT0001-osp-MDT0000: Connection to lustre-MDT0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [12052.473074] Lustre: lustre-MDT0001: Not available for connect from 0@lo (stopping) [12052.484257] Lustre: Skipped 1 previous similar message [12052.499579] Lustre: Skipped 1 previous similar message [12057.571669] Lustre: lustre-MDT0001: Not available for connect from 0@lo (stopping) [12057.584722] Lustre: Skipped 2 previous similar messages [12062.182115] Lustre: lustre-MDT0001 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [12062.481882] Lustre: server umount lustre-MDT0001 complete [12062.694969] LustreError: 359576:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [12062.741361] LustreError: 359576:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 2 previous similar messages [12067.827956] LustreError: 361198:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [12067.850190] LustreError: 361198:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 2 previous similar messages [12074.845744] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [12074.952466] Lustre: MGS: Regenerating lustre-MDT0001 log by user request: rc = 0 [12074.957380] Lustre: Found index 1 for lustre-MDT0001, updating log [12074.997115] Lustre: 358553:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [12075.352572] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [12080.546776] Lustre: DEBUG MARKER: oleg327-server.virtnet: executing set_default_debug -1 all [12080.615160] Lustre: lustre-MDT0001-lwp-OST0000: Connection restored to 0@lo (at 0@lo) [12084.766029] LustreError: 363260:0:(obd_mount.c:234:lustre_start_simple()) lustre-MDT0001-lwp-OST0000 attach error -17 [12084.779602] LustreError: 363260:0:(tgt_mount.c:675:lustre_lwp_setup()) lustre-MDT0001-lwp-OST0000: setup up failed: rc -17 [12084.807079] LustreError: 358557:0:(mgc_request.c:614:do_requeue()) failed processing log: -17 [12084.829707] LustreError: 358557:0:(mgc_request.c:614:do_requeue()) Skipped 1 previous similar message [12084.928692] LustreError: 363263:0:(obd_config.c:761:class_setup()) osp: device 8 already setup: rc = -17 [12084.946560] LustreError: 363263:0:(obd_config.c:866:class_detach()) OBD device 8 still set up [12084.957668] LustreError: 363263:0:(obd_config.c:2102:class_config_llog_handler()) MGC192.168.203.127@tcp: cfg command failed: rc = -17 [12084.967876] Lustre: cmd=cf003 0:lustre-MDT0001-osp-MDT0000 1:lustre-MDT0001_UUID 2:192.168.203.127@tcp [12084.967876] [12387.809930] Lustre: lustre-MDT0001-osp-MDT0000: Connection to lustre-MDT0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [12387.837637] Lustre: lustre-MDT0001: Not available for connect from 0@lo (stopping) [12387.850374] Lustre: Skipped 2 previous similar messages [12393.961676] Lustre: server umount lustre-MDT0001 complete [12396.816353] LustreError: 358560:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 192.168.203.27@tcp (no target). If you are running an HA pair check that the target is mounted on the other server. [12396.870813] LustreError: 358560:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 4 previous similar messages [12409.573321] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [12409.895022] Lustre: MGS: Regenerating lustre-MDT0001 log by user request: rc = 0 [12409.897799] Lustre: Found index 1 for lustre-MDT0001, updating log [12409.966460] Lustre: 358554:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [12410.542525] Lustre: lustre-MDT0001: Imperative Recovery enabled, recovery window shrunk from 60-180 down to 60-180 [12416.019842] Lustre: lustre-MDT0001-lwp-OST0000: Connection restored to 0@lo (at 0@lo) [12416.038069] LustreError: 371463:0:(obd_mount.c:234:lustre_start_simple()) lustre-MDT0001-lwp-OST0000 attach error -17 [12416.052827] LustreError: 371463:0:(tgt_mount.c:675:lustre_lwp_setup()) lustre-MDT0001-lwp-OST0000: setup up failed: rc -17 [12416.079189] LustreError: 358557:0:(mgc_request.c:614:do_requeue()) failed processing log: -17 [12416.116580] LustreError: 358557:0:(mgc_request.c:614:do_requeue()) Skipped 1 previous similar message [12416.301549] LustreError: 371466:0:(lod_lov.c:208:lod_add_device()) lustre-MDT0001-osp-MDT0000: cannot connect to next dev lustre-MDT0001_UUID (-114) [12416.366123] LustreError: 371466:0:(obd_config.c:2102:class_config_llog_handler()) MGC192.168.203.127@tcp: cfg command failed: rc = -114 [12416.396246] Lustre: cmd=cf014 0:lustre-MDT0000-mdtlov 1:lustre-MDT0001_UUID 2:1 3:1 [12416.396246] [12418.798579] Lustre: DEBUG MARKER: oleg327-server.virtnet: executing set_default_debug -1 all [12743.142591] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [12743.162088] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [12743.189091] Lustre: Skipped 1 previous similar message [12743.195952] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [12743.202334] Lustre: Skipped 3 previous similar messages [12746.965345] Lustre: server umount lustre-MDT0000 complete [12748.772236] LustreError: 358566:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [12748.797029] LustreError: 358566:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 10 previous similar messages [12749.865967] LustreError: 358546:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1787803973 with bad export cookie 5512394497708209013 [12749.867545] LustreError: MGC192.168.203.127@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [12749.873078] LustreError: 358546:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 1 previous similar message [12753.890219] Lustre: lustre-MDT0001-lwp-OST0000: Connection to lustre-MDT0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [12753.907131] Lustre: Skipped 2 previous similar messages [12756.267388] Lustre: server umount lustre-MDT0001 complete [12761.150448] Lustre: server umount lustre-OST0000 complete [12770.618411] Lustre: DEBUG MARKER: oleg327-server.virtnet: executing set_hostid [12780.169189] Lustre: DEBUG MARKER: oleg327-server.virtnet: executing load_modules_local [12826.842969] Lustre: DEBUG MARKER: == conf-sanity test 120: cross-target rename should not create bad symlinks ========================================================== 00:14:09 (1787804049) [12837.321828] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [12837.766475] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [12837.844196] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [12838.068286] Lustre: lustre-MDT0000: new disk, initializing [12838.242127] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [12838.313589] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [12845.473112] Lustre: DEBUG MARKER: oleg327-server.virtnet: executing set_default_debug -1 all [12860.083142] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [12860.257788] Lustre: 382165:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [12860.342985] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [12860.347252] Lustre: Skipped 1 previous similar message [12860.459303] Lustre: lustre-MDT0001: new disk, initializing [12860.656251] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [12860.760450] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [12860.775369] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [12866.271403] Lustre: DEBUG MARKER: oleg327-server.virtnet: executing set_default_debug -1 all [12876.657277] Lustre: DEBUG MARKER: oleg327-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [12882.837842] Lustre: DEBUG MARKER: oleg327-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [12891.965772] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [12892.209458] Lustre: lustre-OST0000: new disk, initializing [12892.212885] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [12892.218382] Lustre: 384136:0:(osd_compat.c:1353:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [12892.353362] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [12899.453911] Lustre: DEBUG MARKER: oleg327-server.virtnet: executing set_default_debug -1 all [12901.490708] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [12901.511185] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [12901.667970] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [12908.935236] Lustre: DEBUG MARKER: oleg327-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [12917.168618] Lustre: server umount lustre-MDT0000 complete [12917.221058] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [12917.223247] Lustre: lustre-MDT0000-lwp-OST0000: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [12917.237865] LustreError: 382178:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [12917.243555] Lustre: Skipped 1 previous similar message [12917.272275] LustreError: 382178:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 4 previous similar messages [12920.491120] LustreError: 382157:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1787804144 with bad export cookie 5512394497708210966 [12920.496662] LustreError: MGC192.168.203.127@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [12920.499471] LustreError: 382157:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [12920.501193] LustreError: 382973:0:(osp_object.c:618:osp_attr_get()) lustre-MDT0000-osp-MDT0001: osp_attr_get update error [0x200000401:0x2:0x0]: rc = -5 [12920.501347] LustreError: 382973:0:(client.c:1394:ptlrpc_import_delay_req()) @@@ IMP_CLOSED req@0000000067b62df4 x1874647487651584/t0(0) o1000->lustre-MDT0000-osp-MDT0001@0@lo:24/4 lens 264/4320 e 0 to 0 dl 0 ref 2 fl Rpc:QU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 [12921.019435] Lustre: server umount lustre-MDT0001 complete [12935.398770] Lustre: server umount lustre-OST0000 complete [12965.070626] Lustre: DEBUG MARKER: oleg327-server.virtnet: executing set_hostid [12974.563212] Lustre: DEBUG MARKER: oleg327-server.virtnet: executing load_modules_local [13007.611914] Lustre: DEBUG MARKER: == conf-sanity test complete, duration 12792 sec ========= 00:17:10 (1787804230) [13009.840836] Lustre: DEBUG MARKER: === conf-sanity: start cleanup 00:17:12 (1787804232) === [13013.810436] Lustre: DEBUG MARKER: === conf-sanity: finish cleanup 00:17:16 (1787804236) === [13058.716886] Lustre: DEBUG MARKER: oleg327-server.virtnet: executing unload_modules_local [13061.771755] Key type lgssc unregistered [13062.118059] LNet: 390508:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [13062.127592] LNetError: 390508:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [13062.141950] LNet: Removed LNI 192.168.203.127@tcp [13063.084899] Key type .llcrypt unregistered [13063.087867] Key type ._llcrypt unregistered