[ 0.000000] Linux version 4.18.0rh8.10-debug (green@maintenance) (gcc version 8.5.0 20210514 (Red Hat 8.5.0-26) (GCC)) #2 SMP Mon Jul 14 01:24:22 EDT 2025 [ 0.000000] Command line: rd.shell root=nbd:192.168.200.253:rocky8.10:ext4:ro:-p,-b4096 ro crashkernel=256M panic=1 nomodeset ipmtu=9000 ip=dhcp rd.neednet=1 init_on_free=off mitigations=off console=ttyS1,115200 audit=0 [ 0.000000] x86/fpu: Supporting XSAVE feature 0x001: 'x87 floating point registers' [ 0.000000] x86/fpu: Supporting XSAVE feature 0x002: 'SSE registers' [ 0.000000] x86/fpu: Supporting XSAVE feature 0x004: 'AVX registers' [ 0.000000] x86/fpu: xstate_offset[2]: 576, xstate_sizes[2]: 256 [ 0.000000] x86/fpu: Enabled xstate features 0x7, context size is 832 bytes, using 'standard' format. [ 0.000000] signal: max sigframe size: 1776 [ 0.000000] BIOS-provided physical RAM map: [ 0.000000] BIOS-e820: [mem 0x0000000000000000-0x000000000009fbff] usable [ 0.000000] BIOS-e820: [mem 0x000000000009fc00-0x000000000009ffff] reserved [ 0.000000] BIOS-e820: [mem 0x00000000000f0000-0x00000000000fffff] reserved [ 0.000000] BIOS-e820: [mem 0x0000000000100000-0x00000000bffcdfff] usable [ 0.000000] BIOS-e820: [mem 0x00000000bffce000-0x00000000bfffffff] reserved [ 0.000000] BIOS-e820: [mem 0x00000000feffc000-0x00000000feffffff] reserved [ 0.000000] BIOS-e820: [mem 0x00000000fffc0000-0x00000000ffffffff] reserved [ 0.000000] BIOS-e820: [mem 0x0000000100000000-0x0000000146dfffff] usable [ 0.000000] NX (Execute Disable) protection: active [ 0.000000] SMBIOS 2.8 present. [ 0.000000] DMI: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.17.0-10.fc44 06/10/2025 [ 0.000000] Hypervisor detected: KVM [ 0.000000] kvm-clock: Using msrs 4b564d01 and 4b564d00 [ 0.000000] kvm-clock: using sched offset of 501462200 cycles [ 0.000000] clocksource: kvm-clock: mask: 0xffffffffffffffff max_cycles: 0x1cd42e4dffb, max_idle_ns: 881590591483 ns [ 0.000000] tsc: Detected 2399.998 MHz processor [ 0.000000] last_pfn = 0x146e00 max_arch_pfn = 0x400000000 [ 0.000000] x86/PAT: Configuration [0-7]: WB WC UC- UC WB WP UC- WT [ 0.000000] last_pfn = 0xbffce max_arch_pfn = 0x400000000 [ 0.000000] found SMP MP-table at [mem 0x000f54b0-0x000f54bf] [ 0.000000] RAMDISK: [mem 0xbcc54000-0xbffbffff] [ 0.000000] ACPI: Early table checksum verification disabled [ 0.000000] ACPI: RSDP 0x00000000000F52D0 000014 (v00 BOCHS ) [ 0.000000] ACPI: RSDT 0x00000000BFFE247C 000034 (v01 BOCHS BXPC 00000001 BXPC 00000001) [ 0.000000] ACPI: FACP 0x00000000BFFE2318 000074 (v01 BOCHS BXPC 00000001 BXPC 00000001) [ 0.000000] ACPI: DSDT 0x00000000BFFE0040 0022D8 (v01 BOCHS BXPC 00000001 BXPC 00000001) [ 0.000000] ACPI: FACS 0x00000000BFFE0000 000040 [ 0.000000] ACPI: APIC 0x00000000BFFE238C 000090 (v03 BOCHS BXPC 00000001 BXPC 00000001) [ 0.000000] ACPI: HPET 0x00000000BFFE241C 000038 (v01 BOCHS BXPC 00000001 BXPC 00000001) [ 0.000000] ACPI: WAET 0x00000000BFFE2454 000028 (v01 BOCHS BXPC 00000001 BXPC 00000001) [ 0.000000] ACPI: Reserving FACP table memory at [mem 0xbffe2318-0xbffe238b] [ 0.000000] ACPI: Reserving DSDT table memory at [mem 0xbffe0040-0xbffe2317] [ 0.000000] ACPI: Reserving FACS table memory at [mem 0xbffe0000-0xbffe003f] [ 0.000000] ACPI: Reserving APIC table memory at [mem 0xbffe238c-0xbffe241b] [ 0.000000] ACPI: Reserving HPET table memory at [mem 0xbffe241c-0xbffe2453] [ 0.000000] ACPI: Reserving WAET table memory at [mem 0xbffe2454-0xbffe247b] [ 0.000000] No NUMA configuration found [ 0.000000] Faking a node at [mem 0x0000000000000000-0x0000000146dfffff] [ 0.000000] NODE_DATA(0) allocated [mem 0x1465a3000-0x1465cdfff] [ 0.000000] Reserving 256MB of memory at 2512MB for crashkernel (System RAM: 4205MB) [ 0.000000] Zone ranges: [ 0.000000] DMA [mem 0x0000000000001000-0x0000000000ffffff] [ 0.000000] DMA32 [mem 0x0000000001000000-0x00000000ffffffff] [ 0.000000] Normal [mem 0x0000000100000000-0x0000000146dfffff] [ 0.000000] Device empty [ 0.000000] Movable zone start for each node [ 0.000000] Early memory node ranges [ 0.000000] node 0: [mem 0x0000000000001000-0x000000000009efff] [ 0.000000] node 0: [mem 0x0000000000100000-0x00000000bffcdfff] [ 0.000000] node 0: [mem 0x0000000100000000-0x0000000146dfffff] [ 0.000000] Zeroed struct page in unavailable ranges: 4756 pages [ 0.000000] Initmem setup node 0 [mem 0x0000000000001000-0x0000000146dfffff] [ 0.000000] ACPI: PM-Timer IO Port: 0x608 [ 0.000000] ACPI: LAPIC_NMI (acpi_id[0xff] dfl dfl lint[0x1]) [ 0.000000] IOAPIC[0]: apic_id 0, version 17, address 0xfec00000, GSI 0-23 [ 0.000000] ACPI: INT_SRC_OVR (bus 0 bus_irq 0 global_irq 2 dfl dfl) [ 0.000000] ACPI: INT_SRC_OVR (bus 0 bus_irq 5 global_irq 5 high level) [ 0.000000] ACPI: INT_SRC_OVR (bus 0 bus_irq 9 global_irq 9 high level) [ 0.000000] ACPI: INT_SRC_OVR (bus 0 bus_irq 10 global_irq 10 high level) [ 0.000000] ACPI: INT_SRC_OVR (bus 0 bus_irq 11 global_irq 11 high level) [ 0.000000] Using ACPI (MADT) for SMP configuration information [ 0.000000] ACPI: HPET id: 0x8086a201 base: 0xfed00000 [ 0.000000] TSC deadline timer available [ 0.000000] smpboot: Allowing 4 CPUs, 0 hotplug CPUs [ 0.000000] kvm-guest: KVM setup pv remote TLB flush [ 0.000000] kvm-guest: setup PV sched yield [ 0.000000] PM: Registered nosave memory: [mem 0x00000000-0x00000fff] [ 0.000000] PM: Registered nosave memory: [mem 0x0009f000-0x0009ffff] [ 0.000000] PM: Registered nosave memory: [mem 0x000a0000-0x000effff] [ 0.000000] PM: Registered nosave memory: [mem 0x000f0000-0x000fffff] [ 0.000000] PM: Registered nosave memory: [mem 0xbffce000-0xbfffffff] [ 0.000000] PM: Registered nosave memory: [mem 0xc0000000-0xfeffbfff] [ 0.000000] PM: Registered nosave memory: [mem 0xfeffc000-0xfeffffff] [ 0.000000] PM: Registered nosave memory: [mem 0xff000000-0xfffbffff] [ 0.000000] PM: Registered nosave memory: [mem 0xfffc0000-0xffffffff] [ 0.000000] [mem 0xc0000000-0xfeffbfff] available for PCI devices [ 0.000000] Booting paravirtualized kernel on KVM [ 0.000000] clocksource: refined-jiffies: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 1910969940391419 ns [ 0.000000] setup_percpu: NR_CPUS:8192 nr_cpumask_bits:4 nr_cpu_ids:4 nr_node_ids:1 [ 0.000000] percpu: Embedded 63 pages/cpu s221184 r8192 d28672 u524288 [ 0.000000] kvm-guest: PV spinlocks enabled [ 0.000000] PV qspinlock hash table entries: 256 (order: 0, 4096 bytes, linear) [ 0.000000] Built 1 zonelists, mobility grouping on. Total pages: 1059606 [ 0.000000] Policy zone: Normal [ 0.000000] Kernel command line: rd.shell root=nbd:192.168.200.253:rocky8.10:ext4:ro:-p,-b4096 ro crashkernel=256M panic=1 nomodeset ipmtu=9000 ip=dhcp rd.neednet=1 init_on_free=off mitigations=off console=ttyS1,115200 audit=0 [ 0.000000] Specific versions of hardware are certified with Red Hat Enterprise Linux 8. Please see the list of hardware certified with Red Hat Enterprise Linux 8 at https://catalog.redhat.com. [ 0.000000] audit: disabled (until reboot) [ 0.000000] software IO TLB: area num 4. [ 0.000000] Memory: 2829652K/4306352K available (18435K kernel code, 11221K rwdata, 7248K rodata, 2908K init, 18040K bss, 524580K reserved, 0K cma-reserved) [ 0.000000] SLUB: HWalign=64, Order=0-3, MinObjects=0, CPUs=4, Nodes=1 [ 0.000000] kmemleak: Kernel memory leak detector disabled [ 0.000000] ftrace: allocating 41240 entries in 162 pages [ 0.000000] ftrace: allocated 162 pages with 3 groups [ 0.000000] rcu: Hierarchical RCU implementation. [ 0.000000] rcu: RCU event tracing is enabled. [ 0.000000] rcu: RCU restricting CPUs from NR_CPUS=8192 to nr_cpu_ids=4. [ 0.000000] rcu: RCU callback double-/use-after-free debug enabled. [ 0.000000] Rude variant of Tasks RCU enabled. [ 0.000000] Tracing variant of Tasks RCU enabled. [ 0.000000] rcu: RCU calculated value of scheduler-enlistment delay is 100 jiffies. [ 0.000000] rcu: Adjusting geometry for rcu_fanout_leaf=16, nr_cpu_ids=4 [ 0.000000] NR_IRQS: 524544, nr_irqs: 456, preallocated irqs: 16 [ 0.000000] random: get_random_bytes called from start_kernel+0x622/0x9a8 with crng_init=0 [ 0.001000] Console: colour *CGA 80x25 [ 0.001000] printk: console [ttyS1] enabled [ 0.001000] ACPI: Core revision 20220331 [ 0.001000] clocksource: hpet: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 19112604467 ns [ 0.001011] APIC: Switch to symmetric I/O mode setup [ 0.003305] x2apic enabled [ 0.004009] Switched APIC routing to physical x2apic. [ 0.006011] kvm-guest: setup PV IPIs [ 0.009000] ..TIMER: vector=0x30 apic1=0 pin1=2 apic2=-1 pin2=-1 [ 0.009000] clocksource: tsc-early: mask: 0xffffffffffffffff max_cycles: 0x229835b7123, max_idle_ns: 440795242976 ns [ 0.009023] Calibrating delay loop (skipped) preset value.. 4799.99 BogoMIPS (lpj=2399998) [ 0.010012] pid_max: default: 32768 minimum: 301 [ 0.011132] LSM: Security Framework initializing [ 0.012061] Yama: becoming mindful. [ 0.013046] SELinux: Initializing. [ 0.014077] *** VALIDATE selinux *** [ 0.022710] Dentry cache hash table entries: 1048576 (order: 11, 8388608 bytes, vmalloc) [ 0.026860] Inode-cache hash table entries: 524288 (order: 10, 4194304 bytes, vmalloc) [ 0.027178] Mount-cache hash table entries: 16384 (order: 5, 131072 bytes, vmalloc) [ 0.028107] Mountpoint-cache hash table entries: 16384 (order: 5, 131072 bytes, vmalloc) [ 0.030057] *** VALIDATE tmpfs *** [ 0.031378] *** VALIDATE proc *** [ 0.033061] *** VALIDATE cgroup *** [ 0.034010] *** VALIDATE cgroup2 *** [ 0.035270] x86/cpu: User Mode Instruction Prevention (UMIP) activated [ 0.036178] Last level iTLB entries: 4KB 0, 2MB 0, 4MB 0 [ 0.037007] Last level dTLB entries: 4KB 0, 2MB 0, 4MB 0, 1GB 0 [ 0.038026] Spectre V2 : User space: Vulnerable [ 0.039008] Speculative Store Bypass: Vulnerable [ 0.042277] debug: unmapping init [mem 0xffffffffbb059000-0xffffffffbb060fff] [ 0.044803] smpboot: CPU0: Intel(R) Xeon(R) CPU E5-2695 v2 @ 2.40GHz (family: 0x6, model: 0x3e, stepping: 0x4) [ 0.045705] Performance Events: IvyBridge events, full-width counters, Intel PMU driver. [ 0.046019] ... version: 2 [ 0.046969] ... bit width: 48 [ 0.047014] ... generic registers: 4 [ 0.048013] ... value mask: 0000ffffffffffff [ 0.049014] ... max period: 00007fffffffffff [ 0.050014] ... fixed-purpose events: 3 [ 0.051012] ... event mask: 000000070000000f [ 0.052288] rcu: Hierarchical SRCU implementation. [ 0.054488] smp: Bringing up secondary CPUs ... [ 0.055649] x86: Booting SMP configuration: [ 0.056026] .... node #0, CPUs: #1 #2 #3 [ 0.059541] smp: Brought up 1 node, 4 CPUs [ 0.061015] smpboot: Max logical packages: 1 [ 0.062020] smpboot: Total of 4 processors activated (19199.98 BogoMIPS) [ 0.140458] node 0 deferred pages initialised in 76ms [ 0.144099] devtmpfs: initialized [ 0.145270] x86/mm: Memory block size: 128MB [ 0.148878] gcov: version magic: 0x41383552 [ 0.152249] clocksource: jiffies: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 1911260446275000 ns [ 0.156130] futex hash table entries: 1024 (order: 4, 65536 bytes, vmalloc) [ 0.159382] pinctrl core: initialized pinctrl subsystem [ 0.162197] [ 0.162945] ************************************************************* [ 0.166016] ** NOTICE NOTICE NOTICE NOTICE NOTICE NOTICE NOTICE ** [ 0.169011] ** ** [ 0.172011] ** IOMMU DebugFS SUPPORT HAS BEEN ENABLED IN THIS KERNEL ** [ 0.175012] ** ** [ 0.178014] ** This means that this kernel is built to expose internal ** [ 0.181012] ** IOMMU data structures, which may compromise security on ** [ 0.184010] ** your system. ** [ 0.186011] ** ** [ 0.189013] ** If you see this message and you are not debugging the ** [ 0.192012] ** kernel, report this immediately to your vendor! ** [ 0.195011] ** ** [ 0.198014] ** NOTICE NOTICE NOTICE NOTICE NOTICE NOTICE NOTICE ** [ 0.201015] ************************************************************* [ 0.204732] NET: Registered protocol family 16 [ 0.207428] DMA: preallocated 512 KiB GFP_KERNEL pool for atomic allocations [ 0.210075] DMA: preallocated 512 KiB GFP_KERNEL|GFP_DMA pool for atomic allocations [ 0.213069] DMA: preallocated 512 KiB GFP_KERNEL|GFP_DMA32 pool for atomic allocations [ 0.217735] cpuidle: using governor menu [ 0.219834] acpiphp: ACPI Hot Plug PCI Controller Driver version: 0.5 [ 0.224018] PCI: Using configuration type 1 for base access [ 0.226122] core: PMU erratum BJ122, BV98, HSD29 worked around, HT is on [ 0.237399] HugeTLB registered 1.00 GiB page size, pre-allocated 0 pages [ 0.238019] HugeTLB registered 2.00 MiB page size, pre-allocated 0 pages [ 0.240088] cryptd: max_cpu_qlen set to 1000 [ 0.244866] ACPI: Added _OSI(Module Device) [ 0.245016] ACPI: Added _OSI(Processor Device) [ 0.246026] ACPI: Added _OSI(3.0 _SCP Extensions) [ 0.248019] ACPI: Added _OSI(Processor Aggregator Device) [ 0.253532] ACPI: 1 ACPI AML tables successfully acquired and loaded [ 0.260417] ACPI: Interpreter enabled [ 0.262066] ACPI: PM: (supports S0 S3 S4 S5) [ 0.264016] ACPI: Using IOAPIC for interrupt routing [ 0.267094] PCI: Using host bridge windows from ACPI; if necessary, use "pci=nocrs" and report a bug [ 0.271361] ACPI: Enabled 2 GPEs in block 00 to 0F [ 0.282455] ACPI: PCI Root Bridge [PCI0] (domain 0000 [bus 00-ff]) [ 0.285044] acpi PNP0A03:00: _OSC: OS supports [ASPM ClockPM Segments MSI HPX-Type3] [ 0.288020] acpi PNP0A03:00: _OSC: not requesting OS control; OS requires [ExtendedConfig ASPM ClockPM MSI] [ 0.291079] acpi PNP0A03:00: fail to add MMCONFIG information, can't access extended PCI configuration space under this bridge. [ 0.297351] acpiphp: Slot [2] registered [ 0.299192] acpiphp: Slot [5] registered [ 0.302201] acpiphp: Slot [6] registered [ 0.304154] acpiphp: Slot [7] registered [ 0.306140] acpiphp: Slot [8] registered [ 0.307154] acpiphp: Slot [9] registered [ 0.309127] acpiphp: Slot [10] registered [ 0.311137] acpiphp: Slot [3] registered [ 0.313106] acpiphp: Slot [4] registered [ 0.314134] acpiphp: Slot [11] registered [ 0.316123] acpiphp: Slot [12] registered [ 0.318091] acpiphp: Slot [13] registered [ 0.320120] acpiphp: Slot [14] registered [ 0.321120] acpiphp: Slot [15] registered [ 0.323100] acpiphp: Slot [16] registered [ 0.325108] acpiphp: Slot [17] registered [ 0.326108] acpiphp: Slot [18] registered [ 0.328091] acpiphp: Slot [19] registered [ 0.329101] acpiphp: Slot [20] registered [ 0.331115] acpiphp: Slot [21] registered [ 0.333120] acpiphp: Slot [22] registered [ 0.335107] acpiphp: Slot [23] registered [ 0.336113] acpiphp: Slot [24] registered [ 0.338131] acpiphp: Slot [25] registered [ 0.340078] acpiphp: Slot [26] registered [ 0.341075] acpiphp: Slot [27] registered [ 0.342133] acpiphp: Slot [28] registered [ 0.344121] acpiphp: Slot [29] registered [ 0.346108] acpiphp: Slot [30] registered [ 0.347109] acpiphp: Slot [31] registered [ 0.349075] PCI host bridge to bus 0000:00 [ 0.351029] pci_bus 0000:00: root bus resource [io 0x0000-0x0cf7 window] [ 0.354027] pci_bus 0000:00: root bus resource [io 0x0d00-0xffff window] [ 0.356023] pci_bus 0000:00: root bus resource [mem 0x000a0000-0x000bffff window] [ 0.359024] pci_bus 0000:00: root bus resource [mem 0xc0000000-0xfebfffff window] [ 0.362057] pci_bus 0000:00: root bus resource [mem 0xe0000000000-0xe007fffffff window] [ 0.366030] pci_bus 0000:00: root bus resource [bus 00-ff] [ 0.368169] pci 0000:00:00.0: [8086:1237] type 00 class 0x060000 [ 0.372073] pci 0000:00:01.0: [8086:7000] type 00 class 0x060100 [ 0.375358] pci 0000:00:01.1: [8086:7010] type 00 class 0x010180 [ 0.386016] pci 0000:00:01.1: reg 0x20: [io 0xc320-0xc32f] [ 0.391977] pci 0000:00:01.1: legacy IDE quirk: reg 0x10: [io 0x01f0-0x01f7] [ 0.394018] pci 0000:00:01.1: legacy IDE quirk: reg 0x14: [io 0x03f6] [ 0.397018] pci 0000:00:01.1: legacy IDE quirk: reg 0x18: [io 0x0170-0x0177] [ 0.400019] pci 0000:00:01.1: legacy IDE quirk: reg 0x1c: [io 0x0376] [ 0.403649] pci 0000:00:01.3: [8086:7113] type 00 class 0x068000 [ 0.406899] pci 0000:00:01.3: quirk: [io 0x0600-0x063f] claimed by PIIX4 ACPI [ 0.409039] pci 0000:00:01.3: quirk: [io 0x0700-0x070f] claimed by PIIX4 SMB [ 0.412749] pci 0000:00:02.0: [1af4:1000] type 00 class 0x020000 [ 0.419013] pci 0000:00:02.0: reg 0x10: [io 0xc300-0xc31f] [ 0.431014] pci 0000:00:02.0: reg 0x20: [mem 0xe0000000000-0xe0000003fff 64bit pref] [ 0.437012] pci 0000:00:02.0: reg 0x30: [mem 0xfeb80000-0xfebbffff pref] [ 0.445532] pci 0000:00:05.0: [1af4:1001] type 00 class 0x010000 [ 0.456014] pci 0000:00:05.0: reg 0x10: [io 0xc000-0xc07f] [ 0.464016] pci 0000:00:05.0: reg 0x14: [mem 0xfebc0000-0xfebc0fff] [ 0.486015] pci 0000:00:05.0: reg 0x20: [mem 0xe0000004000-0xe0000007fff 64bit pref] [ 0.497182] pci 0000:00:06.0: [1af4:1001] type 00 class 0x010000 [ 0.517013] pci 0000:00:06.0: reg 0x10: [io 0xc080-0xc0ff] [ 0.533016] pci 0000:00:06.0: reg 0x14: [mem 0xfebc1000-0xfebc1fff] [ 0.550016] pci 0000:00:06.0: reg 0x20: [mem 0xe0000008000-0xe000000bfff 64bit pref] [ 0.566603] pci 0000:00:07.0: [1af4:1001] type 00 class 0x010000 [ 0.581025] pci 0000:00:07.0: reg 0x10: [io 0xc100-0xc17f] [ 0.591030] pci 0000:00:07.0: reg 0x14: [mem 0xfebc2000-0xfebc2fff] [ 0.616027] pci 0000:00:07.0: reg 0x20: [mem 0xe000000c000-0xe000000ffff 64bit pref] [ 0.632395] pci 0000:00:08.0: [1af4:1001] type 00 class 0x010000 [ 0.645015] pci 0000:00:08.0: reg 0x10: [io 0xc180-0xc1ff] [ 0.658015] pci 0000:00:08.0: reg 0x14: [mem 0xfebc3000-0xfebc3fff] [ 0.689017] pci 0000:00:08.0: reg 0x20: [mem 0xe0000010000-0xe0000013fff 64bit pref] [ 0.702609] pci 0000:00:09.0: [1af4:1001] type 00 class 0x010000 [ 0.711025] pci 0000:00:09.0: reg 0x10: [io 0xc200-0xc27f] [ 0.720028] pci 0000:00:09.0: reg 0x14: [mem 0xfebc4000-0xfebc4fff] [ 0.746028] pci 0000:00:09.0: reg 0x20: [mem 0xe0000014000-0xe0000017fff 64bit pref] [ 0.758124] pci 0000:00:0a.0: [1af4:1001] type 00 class 0x010000 [ 0.771016] pci 0000:00:0a.0: reg 0x10: [io 0xc280-0xc2ff] [ 0.781015] pci 0000:00:0a.0: reg 0x14: [mem 0xfebc5000-0xfebc5fff] [ 0.804020] pci 0000:00:0a.0: reg 0x20: [mem 0xe0000018000-0xe000001bfff 64bit pref] [ 0.823506] ACPI: PCI: Interrupt link LNKA configured for IRQ 10 [ 0.826568] ACPI: PCI: Interrupt link LNKB configured for IRQ 10 [ 0.829502] ACPI: PCI: Interrupt link LNKC configured for IRQ 11 [ 0.832469] ACPI: PCI: Interrupt link LNKD configured for IRQ 11 [ 0.835277] ACPI: PCI: Interrupt link LNKS configured for IRQ 9 [ 0.840130] iommu: Default domain type: Passthrough [ 0.843304] SCSI subsystem initialized [ 0.845234] ACPI: bus type USB registered [ 0.847149] usbcore: registered new interface driver usbfs [ 0.850158] usbcore: registered new interface driver hub [ 0.852113] usbcore: registered new device driver usb [ 0.854299] pps_core: LinuxPPS API ver. 1 registered [ 0.857031] pps_core: Software ver. 5.3.6 - Copyright 2005-2007 Rodolfo Giometti [ 0.861120] PTP clock support registered [ 0.863130] EDAC MC: Ver: 3.0.0 [ 0.865648] PCI: Using ACPI for IRQ routing [ 0.869052] NetLabel: Initializing [ 0.870011] NetLabel: domain hash size = 128 [ 0.872013] NetLabel: protocols = UNLABELED CIPSOv4 CALIPSO [ 0.875082] NetLabel: unlabeled traffic allowed by default [ 0.878114] vgaarb: loaded [ 0.879468] hpet0: at MMIO 0xfed00000, IRQs 2, 8, 0 [ 0.882028] hpet0: 3 comparators, 64-bit 100.000000 MHz counter [ 0.890200] clocksource: Switched to clocksource kvm-clock [ 0.999704] VFS: Disk quotas dquot_6.6.0 [ 1.001315] VFS: Dquot-cache hash table entries: 512 (order 0, 4096 bytes) [ 1.004585] *** VALIDATE ramfs *** [ 1.005902] *** VALIDATE hugetlbfs *** [ 1.007569] pnp: PnP ACPI init [ 1.009841] pnp: PnP ACPI: found 6 devices [ 1.034559] clocksource: acpi_pm: mask: 0xffffff max_cycles: 0xffffff, max_idle_ns: 2085701024 ns [ 1.038496] pci_bus 0000:00: resource 4 [io 0x0000-0x0cf7 window] [ 1.041691] pci_bus 0000:00: resource 5 [io 0x0d00-0xffff window] [ 1.044524] pci_bus 0000:00: resource 6 [mem 0x000a0000-0x000bffff window] [ 1.047842] pci_bus 0000:00: resource 7 [mem 0xc0000000-0xfebfffff window] [ 1.050277] pci_bus 0000:00: resource 8 [mem 0xe0000000000-0xe007fffffff window] [ 1.053344] NET: Registered protocol family 2 [ 1.056703] IP idents hash table entries: 131072 (order: 8, 1048576 bytes, vmalloc) [ 1.063216] tcp_listen_portaddr_hash hash table entries: 4096 (order: 5, 163840 bytes, vmalloc) [ 1.067949] TCP established hash table entries: 65536 (order: 7, 524288 bytes, vmalloc) [ 1.074555] TCP bind hash table entries: 65536 (order: 9, 2097152 bytes, vmalloc) [ 1.078444] TCP: Hash tables configured (established 65536 bind 65536) [ 1.081966] MPTCP token hash table entries: 8192 (order: 6, 393216 bytes, vmalloc) [ 1.085789] UDP hash table entries: 4096 (order: 6, 393216 bytes, vmalloc) [ 1.088963] UDP-Lite hash table entries: 4096 (order: 6, 393216 bytes, vmalloc) [ 1.092182] NET: Registered protocol family 1 [ 1.095141] RPC: Registered named UNIX socket transport module. [ 1.097626] RPC: Registered udp transport module. [ 1.099664] RPC: Registered tcp transport module. [ 1.101673] RPC: Registered tcp NFSv4.1 backchannel transport module. [ 1.104359] NET: Registered protocol family 44 [ 1.106241] pci 0000:00:00.0: Limiting direct PCI/PCI transfers [ 1.108852] pci 0000:00:01.0: PIIX3: Enabling Passive Release [ 1.111294] pci 0000:00:01.0: Activating ISA DMA hang workarounds [ 1.113149] PCI: CLS 0 bytes, default 64 [ 1.114984] Unpacking initramfs... [ 2.511731] debug: unmapping init [mem 0xffff8bea7cc54000-0xffff8bea7ffbffff] [ 2.515621] PCI-DMA: Using software bounce buffering for IO (SWIOTLB) [ 2.517422] software IO TLB: mapped [mem 0x00000000b8c54000-0x00000000bcc54000] (64MB) [ 2.519665] clocksource: tsc: mask: 0xffffffffffffffff max_cycles: 0x229835b7123, max_idle_ns: 440795242976 ns [ 3.074074] Initialise system trusted keyrings [ 3.075893] Key type blacklist registered [ 3.077893] workingset: timestamp_bits=36 max_order=20 bucket_order=0 [ 3.089706] zbud: loaded [ 3.093775] *** VALIDATE nfs *** [ 3.095083] *** VALIDATE nfs4 *** [ 3.096964] pstore: using deflate compression [ 3.100587] Platform Keyring initialized [ 3.217318] NET: Registered protocol family 38 [ 3.219558] Key type asymmetric registered [ 3.221696] Asymmetric key parser 'x509' registered [ 3.224019] Block layer SCSI generic (bsg) driver version 0.4 loaded (major 247) [ 3.228000] io scheduler mq-deadline registered [ 3.230118] io scheduler kyber registered [ 3.232078] io scheduler bfq registered [ 3.234832] atomic64_test: passed for x86-64 platform with CX8 and with SSE [ 3.238500] shpchp: Standard Hot Plug PCI Controller Driver version: 0.4 [ 3.241810] input: Power Button as /devices/LNXSYSTM:00/LNXPWRBN:00/input/input0 [ 3.245235] ACPI: Power Button [PWRF] [ 3.252763] ACPI: \_SB_.LNKB: Enabled at IRQ 10 [ 3.260487] ACPI: \_SB_.LNKA: Enabled at IRQ 11 [ 3.276167] ACPI: \_SB_.LNKC: Enabled at IRQ 11 [ 3.284235] ACPI: \_SB_.LNKD: Enabled at IRQ 10 [ 3.304942] Serial: 8250/16550 driver, 4 ports, IRQ sharing enabled [ 3.336060] 00:03: ttyS1 at I/O 0x2f8 (irq = 3, base_baud = 115200) is a 16550A [ 3.364389] 00:04: ttyS0 at I/O 0x3f8 (irq = 4, base_baud = 115200) is a 16550A [ 3.369948] Non-volatile memory driver v1.3 [ 3.371937] Linux agpgart interface v0.103 [ 3.406761] virtio_blk virtio1: [vda] 145904 512-byte logical blocks (74.7 MB/71.2 MiB) [ 3.411749] vda: detected capacity change from 0 to 74702848 [ 3.433633] virtio_blk virtio2: [vdb] 2097152 512-byte logical blocks (1.07 GB/1.00 GiB) [ 3.437461] vdb: detected capacity change from 0 to 1073741824 [ 3.457804] virtio_blk virtio3: [vdc] 5120000 512-byte logical blocks (2.62 GB/2.44 GiB) [ 3.461358] vdc: detected capacity change from 0 to 2621440000 [ 3.475328] virtio_blk virtio4: [vdd] 5120000 512-byte logical blocks (2.62 GB/2.44 GiB) [ 3.478099] vdd: detected capacity change from 0 to 2621440000 [ 3.494610] virtio_blk virtio5: [vde] 8388608 512-byte logical blocks (4.29 GB/4.00 GiB) [ 3.497366] vde: detected capacity change from 0 to 4294967296 [ 3.512821] virtio_blk virtio6: [vdf] 8388608 512-byte logical blocks (4.29 GB/4.00 GiB) [ 3.515832] vdf: detected capacity change from 0 to 4294967296 [ 3.523505] libphy: Fixed MDIO Bus: probed [ 3.539101] usbcore: registered new interface driver usbserial_generic [ 3.541982] usbserial: USB Serial support registered for generic [ 3.544550] i8042: PNP: PS/2 Controller [PNP0303:KBD,PNP0f13:MOU] at 0x60,0x64 irq 1,12 [ 3.549431] serio: i8042 KBD port at 0x60,0x64 irq 1 [ 3.551065] serio: i8042 AUX port at 0x60,0x64 irq 12 [ 3.553509] mousedev: PS/2 mouse device common for all mice [ 3.556688] rtc_cmos 00:05: RTC can wake from S4 [ 3.560140] rtc_cmos 00:05: registered as rtc0 [ 3.560235] input: AT Translated Set 2 keyboard as /devices/platform/i8042/serio0/input/input1 [ 3.562223] rtc_cmos 00:05: alarms up to one day, y3k, 242 bytes nvram, hpet irqs [ 3.569664] input: VirtualPS/2 VMware VMMouse as /devices/platform/i8042/serio1/input/input4 [ 3.570149] intel_pstate: CPU model not supported [ 3.575364] input: VirtualPS/2 VMware VMMouse as /devices/platform/i8042/serio1/input/input3 [ 3.580467] hid: raw HID events driver (C) Jiri Kosina [ 3.583059] usbcore: registered new interface driver usbhid [ 3.585525] usbhid: USB HID core driver [ 3.587586] drop_monitor: Initializing network drop monitor service [ 3.590448] Initializing XFRM netlink socket [ 3.593018] NET: Registered protocol family 10 [ 3.596522] Segment Routing with IPv6 [ 3.598184] NET: Registered protocol family 17 [ 3.600568] mpls_gso: MPLS GSO support [ 3.607138] RAS: Correctable Errors collector initialized. [ 3.610087] AVX version of gcm_enc/dec engaged. [ 3.612372] AES CTR mode by8 optimization enabled [ 3.694933] sched_clock: Marking stable (3694910922, 0)->(4628792993, -933882071) [ 3.698764] registered taskstats version 1 [ 3.701573] Loading compiled-in X.509 certificates [ 3.703676] zswap: loaded using pool lzo/zbud [ 3.727866] Key type big_key registered [ 3.742762] Key type encrypted registered [ 3.744635] ima: No TPM chip found, activating TPM-bypass! [ 3.747052] ima: Allocated hash algorithm: sha1 [ 3.749110] ima: No architecture policies found [ 3.750905] evm: Initialising EVM extended attributes: [ 3.753284] evm: security.selinux [ 3.754721] evm: security.ima [ 3.755943] evm: security.capability [ 3.757628] evm: HMAC attrs: 0x1 [ 3.760277] rtc_cmos 00:05: setting system clock to 2026-08-19 05:29:52 UTC (1787117392) [ 3.766842] debug: unmapping init [mem 0xffffffffbc003000-0xffffffffbc1fffff] [ 3.770659] debug: unmapping init [mem 0xffffffffbad82000-0xffffffffbb058fff] [ 3.780070] Write protecting the kernel read-only data: 28672k [ 3.783406] debug: unmapping init [mem 0xffffffffb9403000-0xffffffffb95fffff] [ 3.786241] debug: unmapping init [mem 0xffffffffb9d14000-0xffffffffb9dfffff] [ 3.819314] systemd[1]: systemd 239 (239-82.el8_10.5) running in system mode. (+PAM +AUDIT +SELINUX +IMA -APPARMOR +SMACK +SYSVINIT +UTMP +LIBCRYPTSETUP +GCRYPT +GNUTLS +ACL +XZ +LZ4 +SECCOMP +BLKID +ELFUTILS +KMOD +IDN2 -IDN +PCRE2 default-hierarchy=legacy) [ 3.829564] systemd[1]: Detected virtualization kvm. [ 3.831574] systemd[1]: Detected architecture x86-64. [ 3.833134] systemd[1]: Running in initial RAM disk. Welcome to Rocky Linux 8.10 (Green Obsidian) dracut-049-233.git20240115.el8 (Initramfs)! [ 3.861678] systemd[1]: No hostname configured. [ 3.863624] systemd[1]: Set hostname to . [ 3.865994] random: systemd: uninitialized urandom read (16 bytes read) [ 3.868623] systemd[1]: Initializing machine ID from random generator. [ 4.004269] random: systemd: uninitialized urandom read (16 bytes read) [ 4.007570] systemd[1]: Started Dispatch Password Requests to Console Directory Watch. [ OK ] Started Dispatch Password Requests to Console Directory Watch. [ 4.014639] random: systemd: uninitialized urandom read (16 bytes read) [ 4.017813] systemd[1]: Listening on Journal Socket. [ OK ] Listening on Journal Socket. [ 4.022322] systemd[1]: Reached target Timers. [ OK ] Reached target Timers. [ OK ] Reached target Local File Systems. Starting Create Volatile Files and Directories... [ OK ] Reached target Slices. [ OK ] Listening on udev Control Socket. [ OK ] Listening on Journal Socket (/dev/log). Starting Journal Service... [ OK ] Listening on udev Kernel Socket. Starting Apply Kernel Variables... [ OK ] Reached target Initrd Root Device. [ OK ] Reached target Sockets. [ OK ] Started Memstrack Anylazing Service. [ OK ] Reached target Paths. [ OK ] Reached target Local Encrypted Volumes. Starting Create list of required st…ce nodes for the current kernel... Starting Setup Virtual Console... [ OK ] Reached target Swap. [ OK ] Started Create Volatile Files and Directories. [ OK ] Started Apply Kernel Variables. [ OK ] Started Create list of required sta…vice nodes for the current kernel. Starting Create Static Device Nodes in /dev... [ OK ] Started Setup Virtual Console. [ OK ] Started Create Static Device Nodes in /dev. Starting dracut cmdline hook... [ OK ] Started Journal Service. [ OK ] Started dracut cmdline hook. Starting dracut pre-udev hook... [ 4.649275] device-mapper: uevent: version 1.0.3 [ 4.652113] device-mapper: ioctl: 4.46.0-ioctl (2022-02-22) initialised: dm-devel@redhat.com [ OK ] Started dracut pre-udev hook. Starting udev Kernel Device Manager... [ OK ] Started udev Kernel Device Manager. Starting dracut pre-trigger hook... [ OK ] Started dracut pre-trigger hook. Starting udev Coldplug all Devices... Mounting Kernel Configuration File System... [ OK ] Mounted Kernel Configuration File System. [ OK ] Started udev Coldplug all Devices. Starting dracut initqueue hook... [ OK ] Reached target System Initialization. [ OK ] Reached target Basic System. [ OK ] Started Hardware RNG Entropy Gatherer Daemon. [ 5.436029] random: fast init done [ 5.463272] virtio_net virtio0 ens2: renamed from eth0 [ 5.585746] scsi host0: ata_piix [ 5.633546] scsi host1: ata_piix [ 5.637367] ata1: PATA max MWDMA2 cmd 0x1f0 ctl 0x3f6 bmdma 0xc320 irq 14 [ 5.641017] ata2: PATA max MWDMA2 cmd 0x170 ctl 0x376 bmdma 0xc328 irq 15 [ 9.269310] dracut-initqueue[579]: RTNETLINK answers: File exists [ 10.263927] random: crng init done [ 10.266120] random: 7 urandom warning(s) missed due to ratelimiting Starting nbd nbd0... [ OK ] Started nbd nbd0. [ OK ] Started dracut initqueue hook. [ OK ] Reached target Remote File Systems (Pre). [ OK ] Reached target Remote File Systems. Mounting /sysroot... [ 10.757184] EXT4-fs (nbd0): mounted filesystem with ordered data mode. Opts: (null) [ OK ] Mounted /sysroot. [ OK ] Reached target Initrd Root File System. Starting Reload Configuration from the Real Root... [ OK ] Started Reload Configuration from the Real Root. [ OK ] Reached target Initrd File Systems. [ OK ] Reached target Initrd Default Target. Starting dracut pre-pivot and cleanup hook... [ OK ] Started dracut pre-pivot and cleanup hook. Starting Cleaning Up and Shutting Down Daemons... [ OK ] Stopped dracut pre-pivot and cleanup hook. [ OK ] Stopped target Remote File Systems. [ OK ] Stopped target Remote File Systems (Pre). Stopping Hardware RNG Entropy Gatherer Daemon... [ OK ] Stopped dracut initqueue hook. [ OK ] Stopped target Initrd Default Target. [ OK ] Stopped target Timers. [ OK ] Stopped target Initrd Root Device. [ OK ] Stopped Hardware RNG Entropy Gatherer Daemon. [ OK ] Stopped target Basic System. [ OK ] Stopped target Slices. [ OK ] Stopped target Paths. [ OK ] Stopped target System Initialization. [ OK ] Stopped target Local Encrypted Volumes. [ OK ] Stopped Dispatch Password Requests to Console Directory Watch. [ OK ] Stopped Apply Kernel Variables. [ OK ] Stopped target Swap. [ OK ] Stopped udev Coldplug all Devices. [ OK ] Stopped dracut pre-trigger hook. Stopping udev Kernel Device Manager... [ OK ] Stopped Create Volatile Files and Directories. [ OK ] Stopped target Local File Systems. [ OK ] Stopped target Sockets. [ OK ] Started Cleaning Up and Shutting Down Daemons. [ OK ] Stopped udev Kernel Device Manager. [ OK ] Stopped Create Static Device Nodes in /dev. [ OK ] Stopped Create list of required sta…vice nodes for the current kernel. [ OK ] Stopped dracut pre-udev hook. [ OK ] Stopped dracut cmdline hook. [ OK ] Closed udev Kernel Socket. [ OK ] Closed udev Control Socket. Starting Cleanup udevd DB... [ OK ] Started Cleanup udevd DB. [ OK ] Reached target Switch Root. Starting Switch Root... [ 11.976389] printk: systemd: 26 output lines suppressed due to ratelimiting [ 12.254831] SELinux: Disabled at runtime. [ 12.325198] systemd[1]: systemd 239 (239-82.el8_10.5) running in system mode. (+PAM +AUDIT +SELINUX +IMA -APPARMOR +SMACK +SYSVINIT +UTMP +LIBCRYPTSETUP +GCRYPT +GNUTLS +ACL +XZ +LZ4 +SECCOMP +BLKID +ELFUTILS +KMOD +IDN2 -IDN +PCRE2 default-hierarchy=legacy) [ 12.335816] systemd[1]: Detected virtualization kvm. [ 12.338088] systemd[1]: Detected architecture x86-64. Welcome to Rocky Linux 8.10 (Green Obsidian)! [ 12.870469] systemd[1]: initrd-switch-root.service: Succeeded. [ 12.874302] systemd[1]: Stopped Switch Root. [ OK ] Stopped Switch Root. [ 12.881621] systemd[1]: systemd-journald.service: Service has no hold-off time (RestartSec=0), scheduling restart. [ 12.886405] systemd[1]: systemd-journald.service: Scheduled restart job, restart counter is at 1. [ 12.891168] systemd[1]: Stopped Journal Service. [ OK ] Stopped Journal Service. [ 12.898902] systemd[1]: Starting Journal Service... Starting Journal Service... [ 12.903827] systemd[1]: Listening on udev Kernel Socket. [ OK ] Listening on udev Kernel Socket. [ OK ] Listening on udev Control Socket. [ OK ] Created slice system-getty.slice. [ OK ] Listening on Process Core Dump Socket. [ OK ] Created slice system-sshd\x2dkeygen.slice. Starting udev Coldplug all Devices... Mounting POSIX Message Queue File System... [ OK ] Started Dispatch Password Requests to Console Directory Watch. [ OK ] Listening on RPCbind Server Activation Socket. Starting Create list of required st…ce nodes for the current kernel... [ OK ] Stopped target Switch Root. [ OK ] Stopped target Initrd Root File System. [ OK ] Listening on initctl Compatibility Named Pipe. [FAILED] Failed to set up automount Arbitrar…rmats File System Automount Point. See 'systemctl status proc-sys-fs-binfmt_misc.automount' for details. [ OK ] Created slice system-serial\x2dgetty.slice. [ OK ] Reached target rpc_pipefs.target. [ OK ] Reached target RPC Port Mapper. Mounting Kernel Debug File System... [ OK ] Started Forward Password Requests to Wall Directory Watch. [ OK ] Reached target Local Encrypted Volumes. [ OK ] Reached target Paths. Starting Apply Kernel Variables... Starting Remount Root and Kernel File Systems... [ OK ] Created slice User and Session Slice. [ OK ] Reached target Slices. Activating swap /dev/disk/by-label/SWAP... [ OK ] Stopped target Initrd File Systems. Mounting Huge Pages File System... [ 13.113073] Adding 1048572k swap on /dev/vdb. Priority:-2 extents:1 across:1048572k FS [ OK ] Started Journal Service. [ OK ] Mounted POSIX Message Queue File System. [ OK ] Started Create list of required sta…vice nodes for the current kernel. [ OK ] Mounted Kernel Debug File System. [ OK ] Started Apply Kernel Variables. [FAILED] Failed to start Remount Root and Kernel File Systems. See 'systemctl status systemd-remount-fs.service' for details. [ OK ] Activated swap /dev/disk/by-label/SWAP. [ OK ] Mounted Huge Pages File System. [ OK ] Reached target Swap. Starting Configure read-only root support... Starting Create Static Device Nodes in /dev... Starting Flush Journal to Persistent Storage... [ OK ] Started Flush Journal to Persistent Storage. [ OK ] Started udev Coldplug all Devices. [ OK ] Started Create Static Device Nodes in /dev. Starting udev Kernel Device Manager... [ OK ] Reached target Local File Systems (Pre). Mounting /mnt... Mounting /home/green/git/lustre-release... [ OK ] Mounted /mnt. [ 13.420713] squashfs: version 4.0 (2009/01/31) Phillip Lougher [ OK ] Mounted /home/green/git/lustre-release. [ OK ] Started udev Kernel Device Manager. [ 13.674945] piix4_smbus 0000:00:01.3: SMBus Host Controller at 0x700, revision 0 [ 13.753511] input: PC Speaker as /devices/platform/pcspkr/input/input5 [ 13.912643] RAPL PMU: API unit is 2^-32 Joules, 0 fixed counters, 10737418240 ms ovfl timer [ 13.939794] EDAC sbridge: Ver: 1.1.2 [ 15.811847] Key type dns_resolver registered [ 16.116873] NFS: Registering the id_resolver key type [ 16.119067] Key type id_resolver registered [ 16.120633] Key type id_legacy registered [ OK ] Started Configure read-only root support. Starting Load/Save Random Seed... [ OK ] Reached target Local File Systems. Starting Mark the need to relabel after reboot... Starting Rebuild Dynamic Linker Cache... Starting Create Volatile Files and Directories... [ OK ] Started Load/Save Random Seed. [ OK ] Started Mark the need to relabel after reboot. [ OK ] Started Create Volatile Files and Directories. Starting RPC Bind... Starting Update UTMP about System Boot/Shutdown... [ OK ] Started RPC Bind. [ OK ] Started Update UTMP about System Boot/Shutdown. [ OK ] Started Rebuild Dynamic Linker Cache. Starting Update is Completed... [ OK ] Started Update is Completed. [ OK ] Reached target System Initialization. [ OK ] Listening on D-Bus System Message Bus Socket. [ OK ] Reached target Sockets. [ OK ] Reached target Basic System. [ OK ] Started D-Bus System Message Bus. Starting Network Manager... [ OK ] Started Hardware RNG Entropy Gatherer Daemon. [ OK ] Started irqbalance daemon. [ OK ] Reached target sshd-keygen.target. Starting Restore /run/initramfs on shutdown... Starting Login Service... [ OK ] Started dnf makecache --timer. [ OK ] Started Daily Cleanup of Temporary Directories. [ OK ] Started daily update of the root trust anchor for DNSSEC. [ OK ] Reached target Timers. [ OK ] Started Restore /run/initramfs on shutdown. [ OK ] Started Login Service. [ OK ] Started Network Manager. [ OK ] Reached target Network. Starting Dynamic System Tuning Daemon... Starting GSSAPI Proxy Daemon... Starting OpenSSH server daemon... Starting Network Manager Wait Online... Starting Hostname Service... [ OK ] Started OpenSSH server daemon. [ OK ] Started GSSAPI Proxy Daemon. [ OK ] Reached target NFS client services. [ OK ] Reached target Remote File Systems (Pre). [ OK ] Reached target Remote File Systems. Starting Permit User Sessions... [ OK ] Started Permit User Sessions. [ OK ] Started Getty on tty1. [ OK ] Started Serial Getty on ttyS1. [ OK ] Started Serial Getty on ttyS0. [ OK ] Reached target Login Prompts. [ OK ] Started Command Scheduler. [ OK ] Started Hostname Service. Starting Network Manager Script Dispatcher Service... [ OK ] Started Network Manager Script Dispatcher Service. [ OK ] Started Network Manager Wait Online. [ OK ] Reached target Network is Online. Starting Crash recovery kernel arming... Starting Notify NFS peers of a restart... Starting System Logging Service... [ OK ] Started Notify NFS peers of a restart. [ OK ] Started System Logging Service. Starting Authorization Manager... [ OK ] Started Authorization Manager. [ OK ] Started Dynamic System Tuning Daemon. [ OK ] Reached target Multi-User System. [ OK ] Reached target Graphical Interface. Starting Update UTMP about System Runlevel Changes... [ OK ] Started Update UTMP about System Runlevel Changes. Rocky Linux 8.10 (Green Obsidian) Kernel 4.18.0rh8.10-debug on an x86_64 oleg322-server login: [ 53.458220] libcfs: loading out-of-tree module taints kernel. [ 53.581803] Key type ._llcrypt registered [ 53.584713] Key type .llcrypt registered [ 53.700448] Lustre: DEBUG MARKER: oleg322-server.virtnet: executing set_hostid [ 74.865766] Lustre: DEBUG MARKER: oleg322-server.virtnet: executing load_modules_local [ 75.309208] hrtimer: interrupt took 4196148 ns [ 76.865882] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 76.891361] alg: No test for adler32 (adler32-zlib) [ 78.312918] Lustre: Lustre: Build Version: 2.17.57_1_g9da61ce [ 79.380682] LNet: Added LNI 192.168.203.122@tcp [8/256/0/180] [ 81.184231] Key type lgssc registered [ 83.052035] Lustre: Echo OBD driver; http://www.lustre.org/ [ 101.284687] ZFS: Loaded module v2.3.2-1, ZFS pool version 5000, ZFS filesystem version 5 [ 147.658890] Lustre: DEBUG MARKER: oleg322-server.virtnet: executing load_modules_local [ 160.749976] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 160.779525] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 162.028245] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 162.068184] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 162.216711] Lustre: lustre-MDT0000: new disk, initializing [ 162.357784] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 162.378191] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 167.196540] Lustre: DEBUG MARKER: oleg322-server.virtnet: executing set_default_debug vfstrace rpctrace dlmtrace neterror ha config ioctl super lfsck all [ 181.794708] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 181.938387] Lustre: 6506:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 182.004456] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [ 182.008470] Lustre: Skipped 1 previous similar message [ 182.117056] Lustre: lustre-MDT0001: new disk, initializing [ 182.206503] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 182.254199] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [ 182.275948] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 187.144640] Lustre: DEBUG MARKER: oleg322-server.virtnet: executing set_default_debug vfstrace rpctrace dlmtrace neterror ha config ioctl super lfsck all [ 192.712800] Lustre: Modifying parameter general.debug_raw_pointers=Y in log params [ 203.220202] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 203.467523] Lustre: lustre-OST0000: new disk, initializing [ 203.475259] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 203.483602] Lustre: 8444:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 203.563106] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 204.484418] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [ 204.501761] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 204.595629] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 210.349120] Lustre: DEBUG MARKER: oleg322-server.virtnet: executing set_default_debug vfstrace rpctrace dlmtrace neterror ha config ioctl super lfsck all [ 226.040847] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 226.182642] Lustre: lustre-OST0001: new disk, initializing [ 226.188188] Lustre: srv-lustre-OST0001: No data found on store. Initialize space. [ 226.196976] Lustre: 9516:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 226.313958] Lustre: lustre-OST0001: Imperative Recovery not enabled, recovery window 60-180 [ 232.463470] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x00000002c0000400-0x0000000300000400]:1:ost [ 232.477180] Lustre: cli-lustre-OST0001-super: Allocated super-sequence [0x00000002c0000400-0x0000000300000400]:1:ost] [ 232.535165] Lustre: lustre-OST0001-osc-MDT0000: update sequence from 0x100010000 to 0x2c0000401 [ 233.197889] Lustre: DEBUG MARKER: oleg322-server.virtnet: executing set_default_debug vfstrace rpctrace dlmtrace neterror ha config ioctl super lfsck all [ 245.510870] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 251.760943] Lustre: Setting parameter general.lod.*.mdt_hash=crush in log params [ 258.239465] Lustre: DEBUG MARKER: oleg322-server.virtnet: executing check_logdir /tmp/testlogs/ [ 263.496879] Lustre: DEBUG MARKER: oleg322-server.virtnet: executing yml_node [ 267.692359] Lustre: DEBUG MARKER: Client: 2.17.57.1 [ 270.673960] Lustre: DEBUG MARKER: MDS: 2.17.57.1 [ 273.376160] Lustre: DEBUG MARKER: OSS: 2.17.57.1 [ 275.100214] Lustre: DEBUG MARKER: -----============= acceptance-small: conf-sanity ============----- Wed Aug 19 01:34:21 EDT 2026 [ 292.778851] Lustre: DEBUG MARKER: excepting tests: 102 106 115 32newtarball [ 294.546084] Lustre: DEBUG MARKER: skipping tests SLOW=no: 45 69 106 111 114 [ 314.339231] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 314.354358] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 314.374664] Lustre: Skipped 3 previous similar messages [ 314.393231] Lustre: Skipped 3 previous similar messages [ 319.187549] Lustre: server umount lustre-MDT0000 complete [ 322.986913] LustreError: 8442:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1787117711 with bad export cookie 13953620453370539239 [ 322.987656] LustreError: MGC192.168.203.122@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 323.006943] LustreError: 8442:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 1 previous similar message [ 324.576668] LustreError: 11682:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 324.577680] Lustre: lustre-MDT0001-lwp-OST0000: Connection to lustre-MDT0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 324.607422] LustreError: 11682:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 6 previous similar messages [ 324.625839] Lustre: lustre-MDT0001: Not available for connect from 0@lo (stopping) [ 324.630012] Lustre: Skipped 1 previous similar message [ 327.588570] LustreError: 6513:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 327.588599] Lustre: lustre-MDT0001: Not available for connect from 0@lo (stopping) [ 327.616237] LustreError: 6513:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 1 previous similar message [ 327.650485] Lustre: Skipped 1 previous similar message [ 332.705159] LustreError: 6513:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 332.708807] Lustre: lustre-MDT0001: Not available for connect from 0@lo (stopping) [ 332.737900] LustreError: 6513:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 1 previous similar message [ 332.769783] Lustre: Skipped 1 previous similar message [ 337.376130] Lustre: lustre-MDT0001 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 337.506568] Lustre: server umount lustre-MDT0001 complete [ 341.965511] Lustre: server umount lustre-OST0000 complete [ 347.094813] Lustre: server umount lustre-OST0001 complete [ 355.170636] Lustre: DEBUG MARKER: oleg322-server.virtnet: executing set_hostid [ 366.803794] Lustre: DEBUG MARKER: oleg322-server.virtnet: executing load_modules_local [ 405.245557] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 405.559612] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 405.585754] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 405.694546] Lustre: lustre-MDT0000: new disk, initializing [ 405.770942] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 405.788417] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 410.567708] Lustre: DEBUG MARKER: oleg322-server.virtnet: executing set_default_debug -1 all [ 421.632211] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 421.800742] Lustre: 15072:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 421.889403] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [ 421.895394] Lustre: Skipped 1 previous similar message [ 421.990732] Lustre: lustre-MDT0001: new disk, initializing [ 422.092282] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 422.131356] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [ 422.156654] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 427.360251] Lustre: DEBUG MARKER: oleg322-server.virtnet: executing set_default_debug -1 all [ 437.813565] Lustre: DEBUG MARKER: oleg322-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 444.756993] Lustre: DEBUG MARKER: oleg322-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 453.363053] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 453.601537] Lustre: lustre-OST0000: new disk, initializing [ 453.607485] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 453.611947] Lustre: 17046:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 453.666302] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 455.159355] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [ 455.180359] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 455.343343] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 461.194703] Lustre: DEBUG MARKER: oleg322-server.virtnet: executing set_default_debug -1 all [ 471.549536] Lustre: DEBUG MARKER: oleg322-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 476.981730] Lustre: DEBUG MARKER: oleg322-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 [ 477.257503] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [ 482.610392] Lustre: DEBUG MARKER: oleg322-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid 50 [ 482.914359] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [ 485.860305] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 485.861726] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 485.876312] Lustre: Skipped 1 previous similar message [ 485.882570] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 485.902878] Lustre: Skipped 1 previous similar message [ 491.433320] Lustre: server umount lustre-OST0000 complete [ 496.098343] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 496.105442] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 496.132749] Lustre: Skipped 2 previous similar messages [ 496.146822] Lustre: Skipped 3 previous similar messages [ 501.723937] Lustre: server umount lustre-MDT0000 complete [ 505.425824] LustreError: 15067:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1787117894 with bad export cookie 13953620453370540772 [ 505.431427] LustreError: MGC192.168.203.122@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 505.437804] LustreError: 15067:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 505.843764] Lustre: server umount lustre-MDT0001 complete [ 511.424114] Lustre: DEBUG MARKER: == conf-sanity test 76a: set permanent params with lctl across mounts ========================================================== 01:38:17 (1787117897) [ 518.310631] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 518.964401] LustreError: 19312:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 519.042150] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 524.260197] LustreError: 19313:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 524.276574] LustreError: 19313:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 1 previous similar message [ 524.466114] Lustre: DEBUG MARKER: oleg322-server.virtnet: executing set_default_debug -1 all [ 533.473487] LustreError: 19312:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 533.498905] LustreError: 19312:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 1 previous similar message [ 534.369144] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 539.804426] Lustre: DEBUG MARKER: oleg322-server.virtnet: executing set_default_debug -1 all [ 548.983618] Lustre: DEBUG MARKER: oleg322-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 555.556475] Lustre: DEBUG MARKER: oleg322-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 563.339251] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 570.337325] Lustre: DEBUG MARKER: oleg322-server.virtnet: executing set_default_debug -1 all [ 579.007517] Lustre: DEBUG MARKER: oleg322-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 585.539664] Lustre: Modifying parameter general.osc.*.max_dirty_mb=452 in log params [ 596.448920] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 596.454521] Lustre: lustre-MDT0000-lwp-OST0000: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 596.462168] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 596.462176] Lustre: Skipped 2 previous similar messages [ 596.497167] Lustre: Skipped 1 previous similar message [ 599.333881] Lustre: server umount lustre-MDT0000 complete [ 601.577902] LustreError: 19307:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 601.597140] LustreError: 19307:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 2 previous similar messages [ 602.867945] LustreError: 19294:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1787117991 with bad export cookie 13953620453370541654 [ 602.870350] LustreError: MGC192.168.203.122@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 602.880598] LustreError: 19294:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 603.185638] Lustre: server umount lustre-MDT0001 complete [ 618.332519] Lustre: server umount lustre-OST0000 complete [ 640.476047] Lustre: DEBUG MARKER: oleg322-server.virtnet: executing load_modules_local [ 651.404652] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 651.750937] LustreError: 23695:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 651.828381] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 651.831149] Lustre: Skipped 2 previous similar messages [ 656.758448] Lustre: DEBUG MARKER: oleg322-server.virtnet: executing set_default_debug -1 all [ 666.600502] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 672.123890] Lustre: DEBUG MARKER: oleg322-server.virtnet: executing set_default_debug -1 all [ 675.529767] Lustre: Setting parameter general.debug_raw_pointers=Y in log params [ 681.540186] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 690.667591] Lustre: DEBUG MARKER: oleg322-server.virtnet: executing set_default_debug -1 all [ 702.813438] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 703.031868] Lustre: lustre-OST0001: new disk, initializing [ 703.041556] Lustre: srv-lustre-OST0001: No data found on store. Initialize space. [ 703.049299] Lustre: 26015:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 704.897166] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x00000002c0000400-0x0000000300000400]:1:ost [ 704.916842] Lustre: cli-lustre-OST0001-super: Allocated super-sequence [0x00000002c0000400-0x0000000300000400]:1:ost] [ 705.012753] Lustre: lustre-OST0001-osc-MDT0000: update sequence from 0x100010000 to 0x2c0000401 [ 711.070246] Lustre: DEBUG MARKER: oleg322-server.virtnet: executing set_default_debug -1 all [ 721.798030] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 725.954391] Lustre: Setting parameter general.lod.*.mdt_hash=crush in log params [ 740.843751] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 740.845014] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 740.859190] Lustre: Skipped 4 previous similar messages [ 740.878143] Lustre: Skipped 5 previous similar messages [ 746.640822] Lustre: server umount lustre-MDT0000 complete [ 751.083427] LustreError: 26103:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 751.118657] LustreError: 26103:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 5 previous similar messages [ 751.162301] LustreError: 24837:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1787118139 with bad export cookie 13953620453370542494 [ 751.177596] LustreError: 24837:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 751.180267] LustreError: MGC192.168.203.122@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 751.646599] Lustre: server umount lustre-MDT0001 complete [ 766.968809] Lustre: server umount lustre-OST0000 complete [ 781.924731] Lustre: server umount lustre-OST0001 complete [ 803.081891] Lustre: DEBUG MARKER: oleg322-server.virtnet: executing load_modules_local [ 816.113087] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 816.716333] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 816.719800] Lustre: Skipped 3 previous similar messages [ 821.202399] Lustre: DEBUG MARKER: oleg322-server.virtnet: executing set_default_debug -1 all [ 831.183821] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 837.549259] Lustre: DEBUG MARKER: oleg322-server.virtnet: executing set_default_debug -1 all [ 840.779567] Lustre: 30186:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify general/debug_raw_pointers=Y (mode = 0) failed: rc = -17 [ 848.024614] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 855.823526] Lustre: DEBUG MARKER: oleg322-server.virtnet: executing set_default_debug -1 all [ 865.670976] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 872.907394] Lustre: DEBUG MARKER: oleg322-server.virtnet: executing set_default_debug -1 all [ 881.665359] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 885.280817] Lustre: 32063:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify general/lod.*.mdt_hash=crush (mode = 0) failed: rc = -17 [ 893.409737] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 893.417604] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 893.445737] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 893.448706] Lustre: Skipped 4 previous similar messages [ 898.805754] Lustre: server umount lustre-MDT0000 complete [ 902.119896] LustreError: 29035:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 902.152473] LustreError: 29035:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 9 previous similar messages [ 902.794398] LustreError: 29018:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1787118291 with bad export cookie 13953620453370543796 [ 902.799129] LustreError: MGC192.168.203.122@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 902.806762] LustreError: 29018:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 1 previous similar message [ 903.236518] Lustre: server umount lustre-MDT0001 complete [ 917.780755] Lustre: server umount lustre-OST0000 complete [ 930.707785] Lustre: server umount lustre-OST0001 complete [ 940.448706] Lustre: DEBUG MARKER: == conf-sanity test 76b: verify params log setup correctly ========================================================== 01:45:26 (1787118326) [ 975.406533] Lustre: DEBUG MARKER: oleg322-server.virtnet: executing load_modules_local [ 988.420161] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 992.934563] Lustre: DEBUG MARKER: oleg322-server.virtnet: executing set_default_debug -1 all [ 1003.517954] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1009.916973] Lustre: DEBUG MARKER: oleg322-server.virtnet: executing set_default_debug -1 all [ 1013.372318] Lustre: 36035:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify general/debug_raw_pointers=Y (mode = 0) failed: rc = -17 [ 1021.860044] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 1028.791928] Lustre: DEBUG MARKER: oleg322-server.virtnet: executing set_default_debug -1 all [ 1037.277799] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 1044.660832] Lustre: DEBUG MARKER: oleg322-server.virtnet: executing set_default_debug -1 all [ 1053.760694] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 1058.181699] Lustre: 37913:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify general/lod.*.mdt_hash=crush (mode = 0) failed: rc = -17 [ 1065.442734] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 1065.452922] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 1065.484553] Lustre: Skipped 3 previous similar messages [ 1065.497876] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 1065.503703] Lustre: Skipped 4 previous similar messages [ 1070.929979] Lustre: server umount lustre-MDT0000 complete [ 1074.984447] LustreError: 34866:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1787118463 with bad export cookie 13953620453370544895 [ 1075.000704] LustreError: MGC192.168.203.122@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 1075.004083] LustreError: 34866:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 6 previous similar messages [ 1075.431866] Lustre: server umount lustre-MDT0001 complete [ 1089.863023] Lustre: server umount lustre-OST0000 complete [ 1104.455866] Lustre: server umount lustre-OST0001 complete [ 1112.934951] Lustre: DEBUG MARKER: == conf-sanity test 76c: verify changelog_mask is applied with lctl set_param -P ========================================================== 01:48:19 (1787118499) [ 1130.355367] Lustre: DEBUG MARKER: oleg322-server.virtnet: executing load_modules_local [ 1140.904056] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1141.519674] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 1141.527943] Lustre: Skipped 7 previous similar messages [ 1146.115489] Lustre: DEBUG MARKER: oleg322-server.virtnet: executing set_default_debug -1 all [ 1155.507723] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1160.670643] Lustre: DEBUG MARKER: oleg322-server.virtnet: executing set_default_debug -1 all [ 1164.359606] Lustre: 41205:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify general/debug_raw_pointers=Y (mode = 0) failed: rc = -17 [ 1172.593819] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 1180.354379] Lustre: DEBUG MARKER: oleg322-server.virtnet: executing set_default_debug -1 all [ 1181.228470] LustreError: 41560:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 1181.259265] LustreError: 41560:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 13 previous similar messages [ 1190.461210] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 1198.414684] Lustre: DEBUG MARKER: oleg322-server.virtnet: executing set_default_debug -1 all [ 1206.834399] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 1210.589949] Lustre: 43084:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify general/lod.*.mdt_hash=crush (mode = 0) failed: rc = -17 [ 1211.738637] Lustre: Setting parameter general.mdd.*.changelog_mask=-CLOSE in log params [ 1211.744349] Lustre: Skipped 1 previous similar message [ 1216.483036] Lustre: lustre-MDT0000-lwp-OST0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 1216.510384] Lustre: Skipped 6 previous similar messages [ 1221.141399] Lustre: server umount lustre-MDT0000 complete [ 1225.739391] LustreError: 40036:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1787118614 with bad export cookie 13953620453370546008 [ 1225.744855] LustreError: MGC192.168.203.122@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 1225.755760] LustreError: 40036:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 5 previous similar messages [ 1232.346322] Lustre: server umount lustre-MDT0001 complete [ 1242.464238] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1250.279597] Lustre: Evicted from MGS (at 0@lo) after server handle changed from 0x0 to 0xc1a537df75c6cbbf [ 1250.294041] Lustre: MGC192.168.203.122@tcp: Connection restored to 0@lo (at 0@lo) [ 1256.418939] Lustre: DEBUG MARKER: oleg322-server.virtnet: executing set_default_debug -1 all [ 1261.036320] Lustre: lustre-MDT0000-lwp-OST0001: Connection restored to 0@lo (at 0@lo) [ 1266.410403] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1271.921858] Lustre: DEBUG MARKER: oleg322-server.virtnet: executing set_default_debug -1 all [ 1272.308022] Lustre: lustre-MDT0001-lwp-OST0000: Connection restored to 0@lo (at 0@lo) [ 1272.320660] Lustre: Skipped 1 previous similar message [ 1280.309357] Lustre: DEBUG MARKER: oleg322-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 1282.336960] Lustre: DEBUG MARKER: mdc.lustre-MDT0000-mdc-*.mds_server_uuid in FULL state after 0 sec [ 1288.861722] Lustre: DEBUG MARKER: oleg322-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 1290.542728] Lustre: DEBUG MARKER: mdc.lustre-MDT0001-mdc-*.mds_server_uuid in FULL state after 0 sec [ 1297.379250] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 1303.095902] Lustre: server umount lustre-MDT0000 complete [ 1307.461816] LustreError: 41207:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1787118696 with bad export cookie 13953620453370547135 [ 1307.471604] LustreError: MGC192.168.203.122@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 1307.471816] LustreError: 41207:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 3 previous similar messages [ 1307.987762] Lustre: server umount lustre-MDT0001 complete [ 1322.620256] Lustre: server umount lustre-OST0000 complete [ 1323.296080] Lustre: 3640:0:(client.c:2490:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1787118696/real 1787118696] req@ffff8beaf56c2d80 x1873928486569856/t0(0) o400->lustre-MDT0001-lwp-OST0001@0@lo:12/10 lens 224/224 e 0 to 1 dl 1787118712 ref 1 fl Rpc:XNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 [ 1326.666219] Lustre: server umount lustre-OST0001 complete [ 1337.997345] Lustre: DEBUG MARKER: == conf-sanity test 76d: verify llite.*.xattr_cache can be set by 'lctl set_param -P' correctly ========================================================== 01:52:03 (1787118723) [ 1357.508939] Lustre: DEBUG MARKER: oleg322-server.virtnet: executing load_modules_local [ 1368.642317] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1374.998101] Lustre: DEBUG MARKER: oleg322-server.virtnet: executing set_default_debug -1 all [ 1385.689913] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1391.178697] Lustre: DEBUG MARKER: oleg322-server.virtnet: executing set_default_debug -1 all [ 1394.482266] Lustre: 48848:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify general/debug_raw_pointers=Y (mode = 0) failed: rc = -17 [ 1401.583595] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 1408.933166] Lustre: DEBUG MARKER: oleg322-server.virtnet: executing set_default_debug -1 all [ 1418.020392] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 1426.466972] Lustre: DEBUG MARKER: oleg322-server.virtnet: executing set_default_debug -1 all [ 1435.205085] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 1440.702082] Lustre: Setting parameter general.llite.*.xattr_cache=0 in log params [ 1462.763099] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 1462.775484] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 1462.803480] Lustre: Skipped 7 previous similar messages [ 1462.810880] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 1462.820170] Lustre: Skipped 18 previous similar messages [ 1467.325857] Lustre: server umount lustre-MDT0000 complete [ 1471.396353] LustreError: 47675:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1787118860 with bad export cookie 13953620453370547933 [ 1471.397639] LustreError: MGC192.168.203.122@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 1471.409290] LustreError: 47675:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 2 previous similar messages [ 1471.942481] Lustre: server umount lustre-MDT0001 complete [ 1485.608370] Lustre: server umount lustre-OST0000 complete [ 1498.630724] Lustre: server umount lustre-OST0001 complete [ 1509.356919] Lustre: DEBUG MARKER: == conf-sanity test 77: comma-separated MGS NIDs and failover node NIDs ========================================================== 01:54:55 (1787118895) [ 1512.294438] Lustre: DEBUG MARKER: SKIP: conf-sanity test_77 mixed loopback and real device not working [ 1514.227254] Lustre: DEBUG MARKER: == conf-sanity test 78: run resize2fs on MDT and OST filesystems ========================================================== 01:55:00 (1787118900) [ 1526.610649] Lustre: DEBUG MARKER: oleg322-server.virtnet: executing set_hostid [ 1535.841454] Lustre: DEBUG MARKER: oleg322-server.virtnet: executing load_modules_local [ 1560.888701] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1561.137745] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 1561.240745] Lustre: lustre-MDT0000: new disk, initializing [ 1561.400937] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 1566.489554] Lustre: DEBUG MARKER: oleg322-server.virtnet: executing set_default_debug -1 all [ 1576.845399] Lustre: DEBUG MARKER: oleg322-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 1584.855713] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 1585.176324] Lustre: lustre-OST0000: new disk, initializing [ 1585.183156] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 1585.189172] Lustre: Skipped 1 previous similar message [ 1585.197325] Lustre: 54946:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 1587.053056] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:0:ost [ 1587.065872] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:0:ost] [ 1587.107891] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x240000400 [ 1592.329486] Lustre: DEBUG MARKER: oleg322-server.virtnet: executing set_default_debug -1 all [ 1601.464887] Lustre: DEBUG MARKER: oleg322-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 1603.660751] Lustre: DEBUG MARKER: create test files [ 1643.490678] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 1645.654490] Lustre: server umount lustre-OST0000 complete [ 1650.711357] Lustre: server umount lustre-MDT0000 complete [ 1664.200173] Lustre: DEBUG MARKER: oleg322-server.virtnet: executing unload_modules_local [ 1667.882811] Key type lgssc unregistered [ 1668.407167] LNet: 56585:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 1668.416180] LNetError: 56585:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 1669.481545] LNet: Removed LNI 192.168.203.122@tcp [ 1670.749200] Key type .llcrypt unregistered [ 1670.752550] Key type ._llcrypt unregistered [ 1703.625961] Key type ._llcrypt registered [ 1703.628526] Key type .llcrypt registered [ 1703.796674] Lustre: DEBUG MARKER: oleg322-server.virtnet: executing load_modules_local [ 1704.949589] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 1705.016838] alg: No test for adler32 (adler32-zlib) [ 1706.203743] Lustre: Lustre: Build Version: 2.17.57_1_g9da61ce [ 1706.602438] LNet: Added LNI 192.168.203.122@tcp [8/256/0/180] [ 1708.392247] Key type lgssc registered [ 1709.459780] Lustre: Echo OBD driver; http://www.lustre.org/ [ 1717.954573] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 1717.981855] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1719.298379] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 1724.150939] Lustre: DEBUG MARKER: oleg322-server.virtnet: executing set_default_debug -1 all [ 1732.642608] Lustre: DEBUG MARKER: oleg322-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 1740.359785] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 1740.929669] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 1745.028707] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x240000400:102 to 0x240000400:129) [ 1745.106966] Lustre: 59324:0:(osd_internal.h:1461:osd_trans_exec_op()) lustre-OST0000: opcode 8: before 511 < left 544, rollback = 9 [ 1745.120242] Lustre: 59324:0:(osd_handler.c:2090:osd_trans_dump_creds()) create: 32/128/4, destroy: 0/0/0 [ 1745.130358] Lustre: 59324:0:(osd_handler.c:2097:osd_trans_dump_creds()) attr_set: 0/0/0, xattr_set: 0/0/0 [ 1745.152685] Lustre: 59324:0:(osd_handler.c:2107:osd_trans_dump_creds()) write: 2/18/1, punch: 0/0/0, quota 1/3/0 [ 1745.171653] Lustre: 59324:0:(osd_handler.c:2114:osd_trans_dump_creds()) insert: 32/544/0, delete: 0/0/0 [ 1745.184355] Lustre: 59324:0:(osd_handler.c:2121:osd_trans_dump_creds()) ref_add: 0/0/0, ref_del: 0/0/0 [ 1747.590945] Lustre: DEBUG MARKER: oleg322-server.virtnet: executing set_default_debug -1 all [ 1756.079906] Lustre: DEBUG MARKER: oleg322-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 1761.186504] Lustre: DEBUG MARKER: check files after expanding the MDT and OST filesystems [ 1767.904442] Lustre: DEBUG MARKER: create more files after expanding the MDT and OST filesystems [ 1776.097337] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 1776.137196] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 1778.210970] Lustre: server umount lustre-OST0000 complete [ 1782.599730] Lustre: server umount lustre-MDT0000 complete [ 1794.820976] Lustre: DEBUG MARKER: oleg322-server.virtnet: executing unload_modules_local [ 1797.945914] Key type lgssc unregistered [ 1798.226693] LNet: 60936:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 1798.231874] LNetError: 60936:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 1798.248033] LNet: Removed LNI 192.168.203.122@tcp [ 1799.093182] Key type .llcrypt unregistered [ 1799.095928] Key type ._llcrypt unregistered [ 1832.566872] Key type ._llcrypt registered [ 1832.568946] Key type .llcrypt registered [ 1832.659828] Lustre: DEBUG MARKER: oleg322-server.virtnet: executing load_modules_local [ 1833.957451] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 1833.993906] alg: No test for adler32 (adler32-zlib) [ 1835.098517] Lustre: Lustre: Build Version: 2.17.57_1_g9da61ce [ 1835.373909] LNet: Added LNI 192.168.203.122@tcp [8/256/0/180] [ 1837.112173] Key type lgssc registered [ 1838.211193] Lustre: Echo OBD driver; http://www.lustre.org/ [ 1848.202721] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 1848.253756] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1849.858115] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 1854.647109] Lustre: DEBUG MARKER: oleg322-server.virtnet: executing set_default_debug -1 all [ 1863.021801] Lustre: DEBUG MARKER: oleg322-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 1870.962144] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 1871.507359] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 1874.639243] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x240000400:140 to 0x240000400:161) [ 1874.815867] Lustre: 63774:0:(osd_internal.h:1461:osd_trans_exec_op()) lustre-OST0000: opcode 8: before 510 < left 544, rollback = 9 [ 1874.830294] Lustre: 63774:0:(osd_handler.c:2090:osd_trans_dump_creds()) create: 32/128/5, destroy: 0/0/0 [ 1874.835503] Lustre: 63774:0:(osd_handler.c:2097:osd_trans_dump_creds()) attr_set: 0/0/0, xattr_set: 0/0/0 [ 1874.840751] Lustre: 63774:0:(osd_handler.c:2107:osd_trans_dump_creds()) write: 2/18/1, punch: 0/0/0, quota 1/3/0 [ 1874.846453] Lustre: 63774:0:(osd_handler.c:2114:osd_trans_dump_creds()) insert: 32/544/0, delete: 0/0/0 [ 1874.851816] Lustre: 63774:0:(osd_handler.c:2121:osd_trans_dump_creds()) ref_add: 0/0/0, ref_del: 0/0/0 [ 1879.255413] Lustre: DEBUG MARKER: oleg322-server.virtnet: executing set_default_debug -1 all [ 1888.401536] Lustre: DEBUG MARKER: oleg322-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 1893.502792] Lustre: DEBUG MARKER: check files after shrinking the MDT and OST filesystems [ 1905.633937] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 1905.649174] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 1905.679737] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 1910.159614] Lustre: server umount lustre-OST0000 complete [ 1914.151501] Lustre: server umount lustre-MDT0000 complete [ 1925.490677] Lustre: DEBUG MARKER: oleg322-server.virtnet: executing unload_modules_local [ 1928.300902] Key type lgssc unregistered [ 1928.576513] LNet: 65346:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 1928.590243] LNetError: 65346:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 1928.608661] LNet: Removed LNI 192.168.203.122@tcp [ 1929.434876] Key type .llcrypt unregistered [ 1929.438343] Key type ._llcrypt unregistered [ 1948.069406] Key type ._llcrypt registered [ 1948.080110] Key type .llcrypt registered [ 1948.195250] Lustre: DEBUG MARKER: oleg322-server.virtnet: executing set_hostid [ 1969.196771] Lustre: DEBUG MARKER: oleg322-server.virtnet: executing load_modules_local [ 1970.531231] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 1970.755745] alg: No test for adler32 (adler32-zlib) [ 1971.906929] Lustre: Lustre: Build Version: 2.17.57_1_g9da61ce [ 1972.106286] LNet: Added LNI 192.168.203.122@tcp [8/256/0/180] [ 1973.809969] Key type lgssc registered [ 1975.011897] Lustre: Echo OBD driver; http://www.lustre.org/ [ 2012.295892] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 2012.320580] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 2013.610633] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 2013.643266] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 2013.761719] Lustre: lustre-MDT0000: new disk, initializing [ 2013.874137] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 2013.898701] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 2018.519463] Lustre: DEBUG MARKER: oleg322-server.virtnet: executing set_default_debug -1 all [ 2029.035530] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 2029.162329] Lustre: 68503:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 2029.258053] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [ 2029.271773] Lustre: Skipped 1 previous similar message [ 2029.426393] Lustre: lustre-MDT0001: new disk, initializing [ 2029.568383] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 2029.608514] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [ 2029.649324] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 2034.315481] Lustre: DEBUG MARKER: oleg322-server.virtnet: executing set_default_debug -1 all [ 2044.311301] Lustre: DEBUG MARKER: oleg322-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 2051.304910] Lustre: DEBUG MARKER: oleg322-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 2058.442517] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 2058.707203] Lustre: lustre-OST0000: new disk, initializing [ 2058.711265] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 2058.718731] Lustre: 70472:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 2058.810916] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 2065.247748] Lustre: DEBUG MARKER: oleg322-server.virtnet: executing set_default_debug -1 all [ 2068.613221] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [ 2068.627508] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 2068.728050] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 2075.528794] Lustre: DEBUG MARKER: oleg322-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 2080.501949] Lustre: DEBUG MARKER: oleg322-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 [ 2080.755175] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [ 2086.096590] Lustre: DEBUG MARKER: oleg322-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid 50 [ 2086.352440] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [ 2089.441168] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 2089.448469] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2089.450400] LustreError: Skipped 1 previous similar message [ 2089.451374] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 2089.486892] Lustre: Skipped 1 previous similar message [ 2094.633482] Lustre: server umount lustre-OST0000 complete [ 2099.687554] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2099.690444] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 2099.715307] Lustre: Skipped 1 previous similar message [ 2099.728313] Lustre: Skipped 2 previous similar messages [ 2104.883985] Lustre: server umount lustre-MDT0000 complete [ 2108.790903] LustreError: 68497:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1787119497 with bad export cookie 9075999818301250283 [ 2108.797451] LustreError: MGC192.168.203.122@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 2108.805796] LustreError: 68497:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 1 previous similar message [ 2109.274468] Lustre: server umount lustre-MDT0001 complete [ 2119.429638] Lustre: DEBUG MARKER: == conf-sanity test 79: format MDT/OST without mgs option (should return errors) ========================================================== 02:05:05 (1787119505) [ 2132.211243] Lustre: DEBUG MARKER: oleg322-server.virtnet: executing load_modules_local [ 2177.392317] Lustre: DEBUG MARKER: oleg322-server.virtnet: executing set_hostid [ 2190.141419] Lustre: DEBUG MARKER: oleg322-server.virtnet: executing load_modules_local [ 2228.626721] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 2228.913223] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 2228.959329] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 2229.060653] Lustre: lustre-MDT0000: new disk, initializing [ 2229.188660] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 2229.234425] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 2234.019645] Lustre: DEBUG MARKER: oleg322-server.virtnet: executing set_default_debug -1 all [ 2244.796126] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 2244.907622] Lustre: 76492:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 2244.977825] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [ 2244.985265] Lustre: Skipped 1 previous similar message [ 2245.133328] Lustre: lustre-MDT0001: new disk, initializing [ 2245.236725] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 2245.283128] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [ 2245.292417] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 2250.059465] Lustre: DEBUG MARKER: oleg322-server.virtnet: executing set_default_debug -1 all [ 2260.573265] Lustre: DEBUG MARKER: oleg322-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 2267.518120] Lustre: DEBUG MARKER: oleg322-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 2275.161946] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 2275.522556] Lustre: lustre-OST0000: new disk, initializing [ 2275.526166] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 2275.532861] Lustre: 78460:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 2275.665368] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 2282.381749] Lustre: DEBUG MARKER: oleg322-server.virtnet: executing set_default_debug -1 all [ 2283.607270] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [ 2283.619657] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 2283.813350] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 2293.363151] Lustre: DEBUG MARKER: oleg322-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 2299.163897] Lustre: DEBUG MARKER: oleg322-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 [ 2299.356741] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [ 2304.122448] Lustre: DEBUG MARKER: oleg322-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid 50 [ 2304.313794] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [ 2309.612708] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2309.613244] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 2309.616512] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 2309.637478] Lustre: Skipped 1 previous similar message [ 2309.672844] LustreError: Skipped 1 previous similar message [ 2312.410108] Lustre: server umount lustre-OST0000 complete [ 2316.770077] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 2316.782650] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2316.797359] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 2316.803879] Lustre: Skipped 1 previous similar message [ 2324.962211] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 2324.976619] Lustre: Skipped 3 previous similar messages [ 2330.081808] Lustre: lustre-MDT0000 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 2330.282992] Lustre: server umount lustre-MDT0000 complete [ 2333.722476] LustreError: 76486:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1787119722 with bad export cookie 9075999818301251165 [ 2333.727184] LustreError: MGC192.168.203.122@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 2333.755468] LustreError: 76486:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 6 previous similar messages [ 2334.049188] Lustre: server umount lustre-MDT0001 complete [ 2342.987818] Lustre: DEBUG MARKER: == conf-sanity test 80: mgc import reconnect race ======== 02:08:49 (1787119729) [ 2348.879531] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 2349.308376] LustreError: 80877:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 2349.332511] LustreError: 80877:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 3 previous similar messages [ 2349.396863] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 2353.533520] Lustre: DEBUG MARKER: oleg322-server.virtnet: executing set_default_debug -1 all [ 2354.657748] LustreError: 80878:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 2359.778465] LustreError: 80877:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 2361.081443] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 2366.292593] Lustre: DEBUG MARKER: oleg322-server.virtnet: executing set_default_debug -1 all [ 2373.864837] Lustre: DEBUG MARKER: oleg322-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 2380.348452] Lustre: DEBUG MARKER: oleg322-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 2388.495147] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 2388.941848] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 2388.958597] Lustre: Skipped 1 previous similar message [ 2395.658708] Lustre: DEBUG MARKER: oleg322-server.virtnet: executing set_default_debug -1 all [ 2404.246131] Lustre: DEBUG MARKER: oleg322-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 2407.721106] Lustre: 83549:0:(genops.c:1795:obd_export_evict_by_uuid()) MGS: evicting 8fdb02b6-73fa-4c17-8bae-fcb80e09fbe2 at adminstrative request [ 2411.493382] LustreError: MGC192.168.203.122@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 2411.516688] Lustre: *** cfs_fail_loc=906, val=2147483648*** [ 2421.739595] Lustre: MGS: Client 8fdb02b6-73fa-4c17-8bae-fcb80e09fbe2 (at 0@lo) reconnecting [ 2421.748294] Lustre: Evicted from MGS (at 0@lo) after server handle changed from 0x7df46dbf7f5b21cf to 0x7df46dbf7f5b246f [ 2421.763248] Lustre: MGC192.168.203.122@tcp: Connection restored to 0@lo (at 0@lo) [ 2444.866944] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 2445.088690] Lustre: lustre-OST0001: new disk, initializing [ 2445.098767] Lustre: srv-lustre-OST0001: No data found on store. Initialize space. [ 2445.107579] Lustre: 83943:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 2451.002515] Lustre: DEBUG MARKER: oleg322-server.virtnet: executing set_default_debug -1 all [ 2454.182476] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x00000002c0000400-0x0000000300000400]:1:ost [ 2454.200749] Lustre: cli-lustre-OST0001-super: Allocated super-sequence [0x00000002c0000400-0x0000000300000400]:1:ost] [ 2454.365432] Lustre: lustre-OST0001-osc-MDT0000: update sequence from 0x100010000 to 0x2c0000401 [ 2460.770422] Lustre: DEBUG MARKER: oleg322-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0001-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 2464.739742] Lustre: lustre-OST0001-osc-MDT0001: Connection to lustre-OST0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2464.749826] Lustre: lustre-OST0001: Not available for connect from 0@lo (stopping) [ 2464.750000] LustreError: lustre-OST0001-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 2464.752193] Lustre: Skipped 2 previous similar messages [ 2464.767281] Lustre: Skipped 2 previous similar messages [ 2469.628982] Lustre: server umount lustre-OST0001 complete [ 2469.872429] LustreError: 82697:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 2469.898794] LustreError: 82697:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 1 previous similar message [ 2474.994498] LustreError: 82699:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 2475.008936] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2475.024651] LustreError: 82699:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 1 previous similar message [ 2479.476626] Lustre: server umount lustre-OST0000 complete [ 2484.202965] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 2484.219085] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 2484.227589] Lustre: Skipped 3 previous similar messages [ 2489.665526] Lustre: server umount lustre-MDT0000 complete [ 2490.338106] LustreError: 80873:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 2490.368463] LustreError: 80873:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 1 previous similar message [ 2493.587391] LustreError: 82719:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1787119882 with bad export cookie 9075999818301252719 [ 2493.592387] LustreError: MGC192.168.203.122@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 2493.608039] LustreError: 82719:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 2494.108453] Lustre: server umount lustre-MDT0001 complete [ 2503.564659] Lustre: DEBUG MARKER: == conf-sanity test 81: sparse OST indexing ============== 02:11:29 (1787119889) [ 2505.077601] Lustre: DEBUG MARKER: SKIP: conf-sanity test_81 needs >= 3 OSTs [ 2506.919852] Lustre: DEBUG MARKER: == conf-sanity test 82a: specify OSTs for file (succeed) or directory (succeed) ========================================================== 02:11:33 (1787119893) [ 2508.532289] Lustre: DEBUG MARKER: SKIP: conf-sanity test_82a needs >= 3 OSTs [ 2510.576833] Lustre: DEBUG MARKER: == conf-sanity test 82b: specify OSTs for file with --pool and --ost-list options ========================================================== 02:11:36 (1787119896) [ 2512.314465] Lustre: DEBUG MARKER: SKIP: conf-sanity test_82b needs >= 4 OSTs [ 2514.578963] Lustre: DEBUG MARKER: == conf-sanity test 82c: specify sparse OSTs for setstripe (should not crash) ========================================================== 02:11:40 (1787119900) [ 2516.277630] Lustre: DEBUG MARKER: SKIP: conf-sanity test_82c needs >= 4 OSTs [ 2518.671305] Lustre: DEBUG MARKER: == conf-sanity test 83: ENOSPACE on OST doesn't cause message VFS: Busy inodes after unmount ... ========================================================== 02:11:44 (1787119904) [ 2520.390550] Lustre: DEBUG MARKER: mount the OST /dev/mapper/ost1_flakey as a ldiskfs filesystem [ 2526.433298] loop: module loaded [ 2526.469181] loop0: detected capacity change from 0 to 4294967296 [ 2526.505827] LDISKFS-fs (loop0): mounted filesystem with ordered data mode. Opts: (null) [ 2528.096517] Lustre: DEBUG MARKER: run llverfs in partial mode on the OST ldiskfs /mnt/lustre-ost1 [ 2531.763199] Lustre: DEBUG MARKER: oleg322-server.virtnet: executing run_llverfs /mnt/lustre-ost1 -vpl no [ 2536.453190] Lustre: DEBUG MARKER: unmount the OST /dev/mapper/ost1_flakey [ 2564.194139] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 2564.242730] LustreError: 88065:0:(obd_config.c:845:class_setup()) setup lustre-OST0000-osd failed (-28) [ 2564.249487] LustreError: 88065:0:(obd_mount.c:259:lustre_start_simple()) lustre-OST0000-osd setup error -28 [ 2564.268415] LustreError: 88065:0:(tgt_mount.c:2579:server_fill_super()) Unable to start osd on /dev/mapper/ost1_flakey: -28 [ 2564.279809] LustreError: 88065:0:(super25.c:179:lustre_fill_super()) llite: Unable to mount : rc = -28 [ 2584.373684] Lustre: DEBUG MARKER: oleg322-server.virtnet: executing set_hostid [ 2595.896933] Lustre: DEBUG MARKER: oleg322-server.virtnet: executing load_modules_local [ 2636.790443] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 2637.050543] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 2637.081409] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 2637.193858] Lustre: lustre-MDT0000: new disk, initializing [ 2637.251400] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 2637.258602] Lustre: Skipped 1 previous similar message [ 2637.277512] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 2642.727084] Lustre: DEBUG MARKER: oleg322-server.virtnet: executing set_default_debug -1 all [ 2654.063569] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 2654.194284] Lustre: 90934:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 2654.451603] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 2659.705534] Lustre: DEBUG MARKER: oleg322-server.virtnet: executing set_default_debug -1 all [ 2672.104606] Lustre: DEBUG MARKER: oleg322-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 2679.798991] Lustre: DEBUG MARKER: oleg322-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 2689.141673] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 2689.305412] Lustre: 92905:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 2695.968707] Lustre: DEBUG MARKER: oleg322-server.virtnet: executing set_default_debug -1 all [ 2697.824807] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 2697.939334] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 2706.069509] Lustre: DEBUG MARKER: oleg322-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 2711.480426] Lustre: DEBUG MARKER: oleg322-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 [ 2711.732462] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [ 2716.644873] Lustre: DEBUG MARKER: oleg322-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid 50 [ 2716.821919] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [ 2723.820917] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 2723.824340] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2723.830691] LustreError: Skipped 1 previous similar message [ 2723.835932] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 2723.835940] Lustre: Skipped 2 previous similar messages [ 2723.891785] Lustre: Skipped 4 previous similar messages [ 2725.054685] Lustre: server umount lustre-OST0000 complete [ 2737.643510] Lustre: server umount lustre-MDT0000 complete [ 2739.169398] LustreError: 90946:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 2739.191788] LustreError: 90946:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 1 previous similar message [ 2742.285394] LustreError: 90926:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1787120131 with bad export cookie 9075999818301253335 [ 2742.297558] LustreError: MGC192.168.203.122@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 2742.300034] LustreError: 90926:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 2 previous similar messages [ 2742.712343] Lustre: server umount lustre-MDT0001 complete [ 2753.059748] Lustre: DEBUG MARKER: == conf-sanity test 84: check recovery_hard_time ========= 02:15:39 (1787120139) [ 2759.665657] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 2765.003799] Lustre: DEBUG MARKER: oleg322-server.virtnet: executing set_default_debug -1 all [ 2773.179962] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 2773.618303] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-60 [ 2773.623409] Lustre: Skipped 3 previous similar messages [ 2778.162493] Lustre: DEBUG MARKER: oleg322-server.virtnet: executing set_default_debug -1 all [ 2786.147833] Lustre: DEBUG MARKER: oleg322-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 2791.511687] Lustre: DEBUG MARKER: oleg322-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 2798.207900] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 2804.277220] Lustre: DEBUG MARKER: oleg322-server.virtnet: executing set_default_debug -1 all [ 2811.392433] Lustre: DEBUG MARKER: oleg322-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 2816.032040] Lustre: DEBUG MARKER: oleg322-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 [ 2816.223781] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [ 2820.278144] Lustre: DEBUG MARKER: oleg322-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid 50 [ 2820.447995] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [ 2826.489899] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 2826.735248] Lustre: lustre-OST0001: new disk, initializing [ 2826.738432] Lustre: Skipped 2 previous similar messages [ 2826.746039] Lustre: srv-lustre-OST0001: No data found on store. Initialize space. [ 2826.753253] Lustre: Skipped 3 previous similar messages [ 2826.759670] Lustre: 98691:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 2832.268453] Lustre: DEBUG MARKER: oleg322-server.virtnet: executing set_default_debug -1 all [ 2832.535972] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x00000002c0000400-0x0000000300000400]:1:ost [ 2832.544475] Lustre: Skipped 2 previous similar messages [ 2832.551783] Lustre: cli-lustre-OST0001-super: Allocated super-sequence [0x00000002c0000400-0x0000000300000400]:1:ost] [ 2832.696206] Lustre: lustre-OST0001-osc-MDT0000: update sequence from 0x100010000 to 0x2c0000401 [ 2841.517898] Lustre: DEBUG MARKER: oleg322-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0001-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 2846.043695] Lustre: DEBUG MARKER: oleg322-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0001-osc-MDT0000.ost_server_uuid 50 [ 2846.235577] Lustre: DEBUG MARKER: os[cp].lustre-OST0001-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [ 2850.715349] Lustre: DEBUG MARKER: oleg322-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0001-osc-MDT0001.ost_server_uuid 50 [ 2850.935698] Lustre: DEBUG MARKER: os[cp].lustre-OST0001-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [ 2870.343491] Lustre: DEBUG MARKER: mds1 REPLAY BARRIER on lustre-MDT0000 [ 2894.812128] Lustre: Failing over lustre-MDT0000 [ 2895.123116] LustreError: 100790:0:(ldlm_resource.c:1207:ldlm_resource_complain()) MGS: namespace resource [0x65727473756c:0x0:0x0].0x0 (0000000029c74d9e) refcount nonzero (2) after lock cleanup; forcing cleanup. [ 2896.353522] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 2896.372773] LustreError: Skipped 1 previous similar message [ 2896.378088] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2896.388333] Lustre: Skipped 2 previous similar messages [ 2896.396440] LustreError: 96048:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 2896.409093] LustreError: 96048:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 3 previous similar messages [ 2897.234327] Lustre: server umount lustre-MDT0000 complete [ 2915.808084] Lustre: 66723:0:(client.c:2490:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1787120288/real 1787120288] req@00000000576e62ba x1873930470304000/t0(0) o400->MGC192.168.203.122@tcp@0@lo:26/25 lens 224/224 e 0 to 1 dl 1787120304 ref 1 fl Rpc:XNQr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 [ 2915.838078] LustreError: MGC192.168.203.122@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 2921.598779] LDISKFS-fs (dm-0): 4 truncates cleaned up [ 2921.603429] LDISKFS-fs (dm-0): recovery complete [ 2921.612905] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 2926.478426] Lustre: lustre-MDT0000: in recovery but waiting for the first client to connect [ 2931.006929] Lustre: lustre-MDT0000: Will be in recovery for at least 1:00, or until 3 clients reconnect [ 2931.007450] Lustre: 95315:0:(ldlm_lib.c:2085:extend_recovery_timer()) lustre-MDT0000: extended recovery timer reached hard limit: 60, extend: 0 [ 2931.689712] Lustre: 96048:0:(ldlm_lib.c:2085:extend_recovery_timer()) lustre-MDT0000: extended recovery timer reached hard limit: 60, extend: 0 [ 2931.698273] Lustre: DEBUG MARKER: oleg322-server.virtnet: executing set_default_debug -1 all [ 2931.706603] Lustre: 96048:0:(ldlm_lib.c:2085:extend_recovery_timer()) Skipped 1 previous similar message [ 2931.712618] Lustre: lustre-MDT0000-lwp-MDT0001: Connection restored to 0@lo (at 0@lo) [ 2931.777470] LustreError: 101537:0:(ldlm_lib.c:2698:replay_request_or_update()) cfs_fail_timeout id 709 sleeping for 300ms [ 2932.123227] LustreError: 101537:0:(ldlm_lib.c:2698:replay_request_or_update()) cfs_fail_timeout id 709 awake [ 2932.463391] LustreError: 101537:0:(ldlm_lib.c:2698:replay_request_or_update()) cfs_fail_timeout id 709 sleeping for 300ms [ 2932.479236] LustreError: 101537:0:(ldlm_lib.c:2698:replay_request_or_update()) Skipped 1 previous similar message [ 2932.811442] LustreError: 101537:0:(ldlm_lib.c:2698:replay_request_or_update()) cfs_fail_timeout id 709 awake [ 2932.823686] LustreError: 101537:0:(ldlm_lib.c:2698:replay_request_or_update()) Skipped 1 previous similar message [ 2932.834233] Lustre: 101537:0:(ldlm_lib.c:2085:extend_recovery_timer()) lustre-MDT0000: extended recovery timer reached hard limit: 60, extend: 1 [ 2932.845279] Lustre: 101537:0:(ldlm_lib.c:2085:extend_recovery_timer()) Skipped 7 previous similar messages [ 2933.492245] LustreError: 101537:0:(ldlm_lib.c:2698:replay_request_or_update()) cfs_fail_timeout id 709 sleeping for 300ms [ 2933.509131] LustreError: 101537:0:(ldlm_lib.c:2698:replay_request_or_update()) Skipped 2 previous similar messages [ 2933.824204] LustreError: 101537:0:(ldlm_lib.c:2698:replay_request_or_update()) cfs_fail_timeout id 709 awake [ 2933.837734] LustreError: 101537:0:(ldlm_lib.c:2698:replay_request_or_update()) Skipped 2 previous similar messages [ 2935.123609] Lustre: 101537:0:(ldlm_lib.c:2085:extend_recovery_timer()) lustre-MDT0000: extended recovery timer reached hard limit: 60, extend: 1 [ 2935.141472] Lustre: 101537:0:(ldlm_lib.c:2085:extend_recovery_timer()) Skipped 6 previous similar messages [ 2935.787718] LustreError: 101537:0:(ldlm_lib.c:2698:replay_request_or_update()) cfs_fail_timeout id 709 sleeping for 300ms [ 2935.800563] LustreError: 101537:0:(ldlm_lib.c:2698:replay_request_or_update()) Skipped 6 previous similar messages [ 2936.121188] LustreError: 101537:0:(ldlm_lib.c:2698:replay_request_or_update()) cfs_fail_timeout id 709 awake [ 2936.126960] LustreError: 101537:0:(ldlm_lib.c:2698:replay_request_or_update()) Skipped 6 previous similar messages [ 2939.297310] Lustre: 101537:0:(ldlm_lib.c:2085:extend_recovery_timer()) lustre-MDT0000: extended recovery timer reached hard limit: 60, extend: 1 [ 2939.315433] Lustre: 101537:0:(ldlm_lib.c:2085:extend_recovery_timer()) Skipped 12 previous similar messages [ 2939.945207] LustreError: 101537:0:(ldlm_lib.c:2698:replay_request_or_update()) cfs_fail_timeout id 709 sleeping for 300ms [ 2939.958055] LustreError: 101537:0:(ldlm_lib.c:2698:replay_request_or_update()) Skipped 12 previous similar messages [ 2940.280129] LustreError: 101537:0:(ldlm_lib.c:2698:replay_request_or_update()) cfs_fail_timeout id 709 awake [ 2940.303639] LustreError: 101537:0:(ldlm_lib.c:2698:replay_request_or_update()) Skipped 12 previous similar messages [ 2947.340483] Lustre: 101537:0:(ldlm_lib.c:2085:extend_recovery_timer()) lustre-MDT0000: extended recovery timer reached hard limit: 60, extend: 1 [ 2947.352179] Lustre: 101537:0:(ldlm_lib.c:2085:extend_recovery_timer()) Skipped 24 previous similar messages [ 2947.989907] LustreError: 101537:0:(ldlm_lib.c:2698:replay_request_or_update()) cfs_fail_timeout id 709 sleeping for 300ms [ 2947.996241] LustreError: 101537:0:(ldlm_lib.c:2698:replay_request_or_update()) Skipped 24 previous similar messages [ 2948.328160] LustreError: 101537:0:(ldlm_lib.c:2698:replay_request_or_update()) cfs_fail_timeout id 709 awake [ 2948.338090] LustreError: 101537:0:(ldlm_lib.c:2698:replay_request_or_update()) Skipped 24 previous similar messages [ 2963.350341] Lustre: 101537:0:(ldlm_lib.c:2085:extend_recovery_timer()) lustre-MDT0000: extended recovery timer reached hard limit: 60, extend: 1 [ 2963.365301] Lustre: 101537:0:(ldlm_lib.c:2085:extend_recovery_timer()) Skipped 49 previous similar messages [ 2964.015654] LustreError: 101537:0:(ldlm_lib.c:2698:replay_request_or_update()) cfs_fail_timeout id 709 sleeping for 300ms [ 2964.034266] LustreError: 101537:0:(ldlm_lib.c:2698:replay_request_or_update()) Skipped 49 previous similar messages [ 2964.360167] LustreError: 101537:0:(ldlm_lib.c:2698:replay_request_or_update()) cfs_fail_timeout id 709 awake [ 2964.377772] LustreError: 101537:0:(ldlm_lib.c:2698:replay_request_or_update()) Skipped 49 previous similar messages [ 2991.120459] Lustre: 101537:0:(ldlm_lib.c:2394:target_recovery_overseer()) lustre-MDT0000 recovery is aborted by hard timeout [ 2991.133408] Lustre: 101537:0:(ldlm_lib.c:2404:target_recovery_overseer()) recovery is aborted, evict exports in recovery [ 2991.140186] Lustre: 101537:0:(genops.c:1622:class_disconnect_stale_exports()) lustre-MDT0000: disconnect stale client c398410a-7b86-4a1f-89e3-92a2b066d9ac@192.168.203.22@tcp [ 2991.148696] Lustre: lustre-MDT0000: disconnecting 1 stale clients [ 2991.154146] Lustre: 101537:0:(ldlm_lib.c:1914:abort_req_replay_queue()) @@@ aborted: req@00000000d987f0e8 x1873930448083712/t0(8589934782) o35->c398410a-7b86-4a1f-89e3-92a2b066d9ac@192.168.203.22@tcp:660/0 lens 392/0 e 0 to 0 dl 1787120390 ref 1 fl Complete:/604/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:0 [ 2991.190081] Lustre: lustre-MDT0000-osd: cancel update llog [0x200000400:0x1:0x0] [ 2991.194459] Lustre: lustre-MDT0000: Denying connection for new client c398410a-7b86-4a1f-89e3-92a2b066d9ac (at 192.168.203.22@tcp), waiting for 3 known clients (2 recovered, 0 in progress, and 1 evicted) to recover in 0:00 [ 2991.217932] Lustre: lustre-MDT0001-osp-MDT0000: cancel update llog [0x240000401:0x1:0x0] [ 2991.241755] Lustre: lustre-MDT0000-osp-MDT0001: Connection restored to 0@lo (at 0@lo) [ 2991.248314] Lustre: lustre-MDT0000: Recovery over after 1:00, of 3 clients 2 recovered and 1 was evicted. [ 2991.250229] Lustre: Skipped 2 previous similar messages [ 2991.285107] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:80 to 0x280000401:577) [ 2991.290387] Lustre: lustre-OST0001: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x2c0000401:49 to 0x2c0000401:545) [ 3004.388840] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 3004.400642] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 3004.412881] Lustre: Skipped 3 previous similar messages [ 3004.417683] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 3004.424891] Lustre: Skipped 4 previous similar messages [ 3010.185979] Lustre: server umount lustre-OST0000 complete [ 3013.610898] LustreError: 99062:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-OST0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 3013.636326] LustreError: 99062:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 43 previous similar messages [ 3019.977382] Lustre: server umount lustre-OST0001 complete [ 3030.409093] Lustre: server umount lustre-MDT0000 complete [ 3034.209222] LustreError: 95301:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1787120422 with bad export cookie 9075999818301304554 [ 3034.211029] LustreError: MGC192.168.203.122@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 3034.219953] LustreError: 95301:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 2 previous similar messages [ 3034.687690] Lustre: server umount lustre-MDT0001 complete [ 3043.531521] Lustre: DEBUG MARKER: == conf-sanity test 85: osd_ost init: fail ea_fid_set ==== 02:20:29 (1787120429) [ 3051.065730] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 3051.077987] Lustre: *** cfs_fail_loc=197, val=0*** [ 3066.723593] LustreError: Failed to get MGS log lustre-sptlrpc and no local copy. [ 3071.840741] LustreError: 103515:0:(mgc_request.c:233:do_config_log_add()) MGC192.168.203.122@tcp: failed processing log, type 4: rc = -110 [ 3076.986330] Lustre: *** cfs_fail_loc=197, val=0*** [ 3076.989683] Lustre: Skipped 65 previous similar messages [ 3097.568321] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 3097.572736] Lustre: Skipped 3 previous similar messages [ 3103.706432] Lustre: DEBUG MARKER: oleg322-server.virtnet: executing set_default_debug -1 all [ 3111.147846] Lustre: DEBUG MARKER: oleg322-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 3113.072568] Lustre: server umount lustre-OST0000 complete [ 3121.400908] Lustre: DEBUG MARKER: == conf-sanity test 86: Replacing mkfs.lustre -G option == 02:21:47 (1787120507) [ 3141.053694] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 3145.532792] Lustre: DEBUG MARKER: oleg322-server.virtnet: executing set_default_debug -1 all [ 3146.725907] LustreError: 105134:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 3146.741724] LustreError: 105134:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 5 previous similar messages [ 3153.196627] Lustre: DEBUG MARKER: oleg322-client.virtnet: executing wait_import_state_mount (FULL|IDLE) mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 3156.714832] LustreError: 105164:0:(osp_object.c:618:osp_attr_get()) lustre-MDT0001-osp-MDT0000: osp_attr_get update error [0x200000009:0x1:0x0]: rc = -5 [ 3156.738387] LustreError: 105164:0:(lod_sub_object.c:917:lod_sub_prep_llog()) lustre-MDT0000-mdtlov: can't get id from catalogs: rc = -5 [ 3156.748912] LustreError: 105164:0:(lod_dev.c:511:lod_sub_recovery_thread()) lustre-MDT0001-osp-MDT0000: get update log duration 15, retries 0, failed: rc = -5 [ 3157.143886] Lustre: server umount lustre-MDT0000 complete [ 3187.328526] Lustre: DEBUG MARKER: oleg322-server.virtnet: executing set_hostid [ 3197.176482] Lustre: DEBUG MARKER: oleg322-server.virtnet: executing load_modules_local [ 3237.746447] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 3238.090276] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 3238.115820] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 3238.212763] Lustre: lustre-MDT0000: new disk, initializing [ 3238.291847] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 3243.644286] Lustre: DEBUG MARKER: oleg322-server.virtnet: executing set_default_debug -1 all [ 3254.836091] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 3254.961977] Lustre: 109337:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 3255.147476] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 3259.999775] Lustre: DEBUG MARKER: oleg322-server.virtnet: executing set_default_debug -1 all [ 3272.023855] Lustre: DEBUG MARKER: oleg322-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 3278.926636] Lustre: DEBUG MARKER: oleg322-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 3285.605326] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 3285.956368] Lustre: 111308:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 3292.436495] Lustre: DEBUG MARKER: oleg322-server.virtnet: executing set_default_debug -1 all [ 3295.447791] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 3302.852630] Lustre: DEBUG MARKER: oleg322-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 3307.482229] Lustre: DEBUG MARKER: oleg322-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 [ 3307.673779] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [ 3312.360699] Lustre: DEBUG MARKER: oleg322-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid 50 [ 3312.572566] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [ 3316.193981] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 3316.196789] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 3316.204510] LustreError: Skipped 5 previous similar messages [ 3316.208618] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 3316.224285] Lustre: Skipped 6 previous similar messages [ 3316.242716] Lustre: Skipped 11 previous similar messages [ 3321.457506] Lustre: server umount lustre-OST0000 complete [ 3332.933376] Lustre: server umount lustre-MDT0000 complete [ 3337.274739] LustreError: 111307:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1787120726 with bad export cookie 9075999818301315271 [ 3337.285130] LustreError: MGC192.168.203.122@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 3337.286185] LustreError: 111307:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 2 previous similar messages [ 3337.704699] Lustre: server umount lustre-MDT0001 complete [ 3346.960338] Lustre: DEBUG MARKER: == conf-sanity test 87: check if MDT inode can hold EAs with N stripes properly ========================================================== 02:25:33 (1787120733) [ 3356.030188] Lustre: DEBUG MARKER: oleg322-server.virtnet: executing unload_modules_local [ 3359.449286] Key type lgssc unregistered [ 3359.913684] LNet: 113800:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 3359.920723] LNetError: 113800:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 3359.971333] LNet: Removed LNI 192.168.203.122@tcp [ 3360.940775] Key type .llcrypt unregistered [ 3360.944964] Key type ._llcrypt unregistered [ 3378.186774] Key type ._llcrypt registered [ 3378.189575] Key type .llcrypt registered [ 3378.306651] Lustre: DEBUG MARKER: oleg322-server.virtnet: executing set_hostid [ 3397.609101] Lustre: DEBUG MARKER: oleg322-server.virtnet: executing load_modules_local [ 3398.929578] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 3399.215429] alg: No test for adler32 (adler32-zlib) [ 3400.400714] Lustre: Lustre: Build Version: 2.17.57_1_g9da61ce [ 3400.773381] LNet: Added LNI 192.168.203.122@tcp [8/256/0/180] [ 3402.401109] Key type lgssc registered [ 3403.787139] Lustre: Echo OBD driver; http://www.lustre.org/ [ 3444.338029] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 3444.371939] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 3445.771333] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 3445.813054] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 3445.927643] Lustre: lustre-MDT0000: new disk, initializing [ 3446.028468] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 3446.066910] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 3450.625703] Lustre: DEBUG MARKER: oleg322-server.virtnet: executing set_default_debug -1 all [ 3460.669974] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 3460.900575] Lustre: lustre-OST0000: new disk, initializing [ 3460.904811] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 3460.908789] Lustre: Skipped 1 previous similar message [ 3460.914717] Lustre: 117978:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 3460.980921] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 3467.035392] Lustre: DEBUG MARKER: oleg322-server.virtnet: executing set_default_debug -1 all [ 3469.969313] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:0:ost [ 3469.975821] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:0:ost] [ 3470.007446] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x240000400 [ 3477.248876] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 3477.431362] Lustre: lustre-OST0001: new disk, initializing [ 3477.434513] Lustre: srv-lustre-OST0001: No data found on store. Initialize space. [ 3477.439579] Lustre: 118839:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 3477.527271] Lustre: lustre-OST0001: Imperative Recovery not enabled, recovery window 60-180 [ 3478.793176] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:1:ost [ 3478.800573] Lustre: cli-lustre-OST0001-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:1:ost] [ 3478.840030] Lustre: lustre-OST0001-osc-MDT0000: update sequence from 0x100010000 to 0x280000400 [ 3483.496949] Lustre: DEBUG MARKER: oleg322-server.virtnet: executing set_default_debug -1 all [ 3494.371662] Lustre: lustre-MDT0000-lwp-OST0000: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 3494.386342] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 3494.399407] Lustre: Skipped 1 previous similar message [ 3497.920552] Lustre: server umount lustre-MDT0000 complete [ 3503.678323] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: (null) [ 3508.927069] LustreError: 117162:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1787120897 with bad export cookie 8883108324034726536 [ 3508.935810] LustreError: MGC192.168.203.122@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 3509.216511] Lustre: server umount lustre-OST0000 complete [ 3513.890543] Lustre: server umount lustre-OST0001 complete [ 3524.867592] Lustre: DEBUG MARKER: == conf-sanity test 88a: tunefs.lustre persists a --mountfsoptions-only change ========================================================== 02:28:31 (1787120911) [ 3544.610982] Lustre: DEBUG MARKER: == conf-sanity test 88b: check the default mount options can be overridden ========================================================== 02:28:51 (1787120931) [ 3554.302262] Lustre: DEBUG MARKER: oleg322-server.virtnet: executing load_modules_local [ 3576.013741] Lustre: DEBUG MARKER: == conf-sanity test 89: check tunefs --param and --erase-param{s} options ========================================================== 02:29:22 (1787120962) [ 3615.208818] Lustre: DEBUG MARKER: oleg322-server.virtnet: executing set_hostid [ 3624.477511] Lustre: DEBUG MARKER: oleg322-server.virtnet: executing load_modules_local [ 3663.802419] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 3664.108852] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 3664.173434] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 3664.289222] Lustre: lustre-MDT0000: new disk, initializing [ 3664.367056] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 3664.381979] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 3669.290394] Lustre: DEBUG MARKER: oleg322-server.virtnet: executing set_default_debug -1 all [ 3681.461339] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 3681.804905] Lustre: 126599:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 3681.996812] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [ 3682.008336] Lustre: Skipped 1 previous similar message [ 3682.157768] Lustre: lustre-MDT0001: new disk, initializing [ 3682.422668] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 3682.499045] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [ 3682.514919] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 3687.303245] Lustre: DEBUG MARKER: oleg322-server.virtnet: executing set_default_debug -1 all [ 3700.675308] Lustre: DEBUG MARKER: oleg322-client.virtnet: executing wait_import_state_mount FULL failover.node=192.0.2.254@tcp0,mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 3707.244361] Lustre: DEBUG MARKER: oleg322-client.virtnet: executing wait_import_state_mount FULL failover.node=192.0.2.254@tcp0,mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 3714.920647] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 3715.206634] Lustre: lustre-OST0000: new disk, initializing [ 3715.209696] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 3715.213983] Lustre: 128571:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 3715.278905] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 3721.702127] Lustre: DEBUG MARKER: oleg322-server.virtnet: executing set_default_debug -1 all [ 3724.871526] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [ 3724.896569] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 3725.056951] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 3731.571457] Lustre: DEBUG MARKER: oleg322-client.virtnet: executing wait_import_state_mount (FULL|IDLE) failover.node=192.0.2.254@tcp0,osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 3735.771141] Lustre: DEBUG MARKER: oleg322-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 [ 3735.950060] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [ 3740.261170] Lustre: DEBUG MARKER: oleg322-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid 50 [ 3740.487891] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [ 3745.768521] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 3745.770552] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 3745.778347] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 3745.780089] LustreError: Skipped 1 previous similar message [ 3745.810041] Lustre: Skipped 2 previous similar messages [ 3748.731834] Lustre: server umount lustre-OST0000 complete [ 3753.953100] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 3753.967532] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 3753.985543] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 3754.003323] Lustre: Skipped 1 previous similar message [ 3756.002364] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 3756.004718] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 3761.124341] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 3761.139650] Lustre: Skipped 2 previous similar messages [ 3767.264526] Lustre: lustre-MDT0000 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 3767.573909] Lustre: server umount lustre-MDT0000 complete [ 3771.485220] LustreError: 126590:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1787121160 with bad export cookie 8883108324034727761 [ 3771.488808] LustreError: MGC192.168.203.122@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 3771.497747] LustreError: 126590:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 3 previous similar messages [ 3771.841677] Lustre: server umount lustre-MDT0001 complete [ 3781.353579] Lustre: DEBUG MARKER: == conf-sanity test 90a: check max_mod_rpcs_in_flight is enforced ========================================================== 02:32:47 (1787121167) [ 3788.504216] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 3789.227875] LustreError: 130982:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 3789.253161] LustreError: 130982:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 3 previous similar messages [ 3789.330773] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 3794.356611] Lustre: DEBUG MARKER: oleg322-server.virtnet: executing set_default_debug -1 all [ 3794.415163] LustreError: 130983:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 3798.564912] LustreError: 130982:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 3803.631261] LustreError: 130983:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 3803.866292] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 3809.411534] Lustre: DEBUG MARKER: oleg322-server.virtnet: executing set_default_debug -1 all [ 3817.852909] Lustre: DEBUG MARKER: oleg322-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 3824.428185] Lustre: DEBUG MARKER: oleg322-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 3831.114063] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 3831.509272] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 3831.513908] Lustre: Skipped 1 previous similar message [ 3838.064467] Lustre: DEBUG MARKER: oleg322-server.virtnet: executing set_default_debug -1 all [ 3846.304740] Lustre: DEBUG MARKER: oleg322-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 3854.101499] Lustre: *** cfs_fail_loc=159, val=0*** [ 3870.587817] Lustre: lustre-MDT0000: Client 93b9d9c6-8ba7-4c10-b59c-420c53e01e2a (at 192.168.203.22@tcp) reconnecting [ 3872.457854] Lustre: *** cfs_fail_loc=159, val=0*** [ 3872.466885] Lustre: Skipped 5 previous similar messages [ 3889.013358] Lustre: lustre-MDT0000: Client 93b9d9c6-8ba7-4c10-b59c-420c53e01e2a (at 192.168.203.22@tcp) reconnecting [ 3895.283518] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 3895.298246] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 3895.308280] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 3895.318729] Lustre: Skipped 3 previous similar messages [ 3896.135408] Lustre: server umount lustre-OST0000 complete [ 3914.208099] Lustre: lustre-MDT0000 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 3914.445651] Lustre: server umount lustre-MDT0000 complete [ 3915.745567] LustreError: 130983:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 3915.759434] LustreError: 130983:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 1 previous similar message [ 3918.255757] LustreError: 133707:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1787121306 with bad export cookie 8883108324034728643 [ 3918.263777] LustreError: MGC192.168.203.122@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 3918.270101] LustreError: 133707:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 2 previous similar messages [ 3918.612259] Lustre: server umount lustre-MDT0001 complete [ 3931.877156] Lustre: DEBUG MARKER: oleg322-server.virtnet: executing unload_modules_local [ 3935.820167] Key type lgssc unregistered [ 3936.186813] LNet: 134769:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 3936.197273] LNetError: 134769:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 3936.226793] LNet: Removed LNI 192.168.203.122@tcp [ 3937.470801] Key type .llcrypt unregistered [ 3937.478081] Key type ._llcrypt unregistered [ 3963.396472] Key type ._llcrypt registered [ 3963.399406] Key type .llcrypt registered [ 3963.504497] Lustre: DEBUG MARKER: oleg322-server.virtnet: executing load_modules_local [ 3964.372862] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 3964.445474] alg: No test for adler32 (adler32-zlib) [ 3965.556718] Lustre: Lustre: Build Version: 2.17.57_1_g9da61ce [ 3965.826936] LNet: Added LNI 192.168.203.122@tcp [8/256/0/180] [ 3967.512161] Key type lgssc registered [ 3968.523348] Lustre: Echo OBD driver; http://www.lustre.org/ [ 3974.804846] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 3974.819928] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 3976.180067] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 3980.570405] Lustre: DEBUG MARKER: oleg322-server.virtnet: executing set_default_debug -1 all [ 3985.385964] LustreError: 136183:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 3985.406276] LustreError: 136183:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 2 previous similar messages [ 3989.591314] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 3989.944541] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 3994.260231] Lustre: DEBUG MARKER: oleg322-server.virtnet: executing set_default_debug -1 all [ 4002.106905] Lustre: DEBUG MARKER: oleg322-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 4007.536536] Lustre: DEBUG MARKER: oleg322-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 4013.730153] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 4014.043970] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 4020.393324] Lustre: DEBUG MARKER: oleg322-server.virtnet: executing set_default_debug -1 all [ 4021.328059] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:42 to 0x280000401:65) [ 4028.789444] Lustre: DEBUG MARKER: oleg322-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 4040.000518] Lustre: *** cfs_fail_loc=159, val=0*** [ 4055.427207] Lustre: lustre-MDT0000: Client 6320fdb7-71b2-4ba2-9bb2-cc97ad4ea742 (at 192.168.203.22@tcp) reconnecting [ 4057.832187] Lustre: *** cfs_fail_loc=159, val=0*** [ 4073.339711] Lustre: lustre-MDT0001: Client 6320fdb7-71b2-4ba2-9bb2-cc97ad4ea742 (at 192.168.203.22@tcp) reconnecting [ 4075.056406] Lustre: *** cfs_fail_loc=159, val=0*** [ 4075.069434] Lustre: Skipped 4 previous similar messages [ 4091.256088] Lustre: lustre-MDT0001: Client 6320fdb7-71b2-4ba2-9bb2-cc97ad4ea742 (at 192.168.203.22@tcp) reconnecting [ 4096.585742] Lustre: 139219:0:(mdt_handler.c:64:mdt_max_mod_rpcs_per_client_set()) max_mod_rpcs_per_client is deprecated, set mdt.*.max_mod_rpcs_in_flight parameter instead [ 4099.293146] Lustre: *** cfs_fail_loc=159, val=0*** [ 4099.304386] Lustre: Skipped 2 previous similar messages [ 4115.827191] Lustre: lustre-MDT0000: Client 0146ff57-5722-46d7-bad1-c259ae3f2d1a (at 192.168.203.22@tcp) reconnecting [ 4117.559715] Lustre: *** cfs_fail_loc=159, val=0*** [ 4117.562455] Lustre: Skipped 13 previous similar messages [ 4133.252607] Lustre: lustre-MDT0000: Client 0146ff57-5722-46d7-bad1-c259ae3f2d1a (at 192.168.203.22@tcp) reconnecting [ 4135.028452] Lustre: 139508:0:(mdt_handler.c:64:mdt_max_mod_rpcs_per_client_set()) max_mod_rpcs_per_client is deprecated, set mdt.*.max_mod_rpcs_in_flight parameter instead [ 4141.028630] LustreError: lustre-OST0000-osc-MDT0001: operation ost_destroy to node 0@lo failed: rc = -107 [ 4141.030183] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4141.039552] LustreError: Skipped 1 previous similar message [ 4141.062194] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 4144.097356] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 4144.103156] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 4144.113489] LustreError: Skipped 2 previous similar messages [ 4144.113859] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4144.142782] Lustre: Skipped 1 previous similar message [ 4146.573921] Lustre: server umount lustre-OST0000 complete [ 4153.825449] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 4153.833297] LustreError: Skipped 7 previous similar messages [ 4153.846155] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4153.868772] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 4157.028502] Lustre: server umount lustre-MDT0000 complete [ 4159.464201] LustreError: 139336:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 4159.491379] LustreError: 139336:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 1 previous similar message [ 4161.227816] LustreError: 136164:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1787121549 with bad export cookie 2260227739597132261 [ 4161.235340] LustreError: MGC192.168.203.122@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 4161.243934] LustreError: 136164:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 3 previous similar messages [ 4161.707917] Lustre: server umount lustre-MDT0001 complete [ 4173.380743] Lustre: DEBUG MARKER: oleg322-server.virtnet: executing unload_modules_local [ 4176.493389] Key type lgssc unregistered [ 4176.840027] LNet: 140564:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 4176.850042] LNetError: 140564:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 4176.873038] LNet: Removed LNI 192.168.203.122@tcp [ 4177.927978] Key type .llcrypt unregistered [ 4177.930860] Key type ._llcrypt unregistered [ 4205.137406] Key type ._llcrypt registered [ 4205.145927] Key type .llcrypt registered [ 4205.312260] Lustre: DEBUG MARKER: oleg322-server.virtnet: executing load_modules_local [ 4206.899141] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 4207.015882] alg: No test for adler32 (adler32-zlib) [ 4208.123346] Lustre: Lustre: Build Version: 2.17.57_1_g9da61ce [ 4208.465576] LNet: Added LNI 192.168.203.122@tcp [8/256/0/180] [ 4210.160316] Key type lgssc registered [ 4211.592709] Lustre: Echo OBD driver; http://www.lustre.org/ [ 4219.441190] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 4219.480518] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 4221.053252] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 4226.163303] Lustre: DEBUG MARKER: oleg322-server.virtnet: executing set_default_debug -1 all [ 4230.638030] LustreError: 141980:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 4230.657447] LustreError: 141980:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 2 previous similar messages [ 4235.507949] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 4235.746916] Lustre: lustre-MDT0001: Not available for connect from 0@lo (not set up) [ 4235.895665] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 4240.292576] Lustre: DEBUG MARKER: oleg322-server.virtnet: executing set_default_debug -1 all [ 4247.488222] Lustre: DEBUG MARKER: oleg322-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 4252.669441] Lustre: DEBUG MARKER: oleg322-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 4259.621413] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 4259.964996] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 4260.988535] Lustre: lustre-OST0000: new connection from lustre-MDT0001-mdtlov (cleaning up unused objects from 0x280000400:72 to 0x280000400:97) [ 4266.101749] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:85 to 0x280000401:129) [ 4266.420036] Lustre: DEBUG MARKER: oleg322-server.virtnet: executing set_default_debug -1 all [ 4273.786373] Lustre: DEBUG MARKER: oleg322-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 4280.328761] Lustre: 144563:0:(mdt_handler.c:64:mdt_max_mod_rpcs_per_client_set()) max_mod_rpcs_per_client is deprecated, set mdt.*.max_mod_rpcs_in_flight parameter instead [ 4283.755530] Lustre: 144611:0:(mdt_handler.c:64:mdt_max_mod_rpcs_per_client_set()) max_mod_rpcs_per_client is deprecated, set mdt.*.max_mod_rpcs_in_flight parameter instead [ 4285.306258] Lustre: Modifying parameter lustre.mdc.lustre-MDT0000-mdc-*.max_rpcs_in_flight=8 in log params [ 4286.422429] Lustre: Setting parameter lustre.mdc.lustre-MDT0000-mdc-*.max_mod_rpcs_in_flight=8 in log params [ 4289.959079] Lustre: 144758:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre/mdc.lustre-MDT0000-mdc-*.max_rpcs_in_flight=8 (mode = 0) failed: rc = -17 [ 4290.875258] Lustre: Modifying parameter lustre.mdc.lustre-MDT0000-mdc-*.max_mod_rpcs_in_flight=7 in log params [ 4291.862468] Lustre: 144857:0:(mdt_handler.c:64:mdt_max_mod_rpcs_per_client_set()) max_mod_rpcs_per_client is deprecated, set mdt.*.max_mod_rpcs_in_flight parameter instead [ 4297.196327] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 4297.208568] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4297.224048] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 4297.228705] Lustre: Skipped 1 previous similar message [ 4297.247877] Lustre: Skipped 1 previous similar message [ 4301.623863] Lustre: server umount lustre-OST0000 complete [ 4307.425695] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4307.449787] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 4307.455189] Lustre: Skipped 1 previous similar message [ 4311.616492] Lustre: server umount lustre-MDT0000 complete [ 4312.545173] LustreError: 143219:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 4312.554994] LustreError: 143219:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 1 previous similar message [ 4314.683653] LustreError: 141962:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1787121703 with bad export cookie 16797771033585517146 [ 4314.695463] LustreError: MGC192.168.203.122@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 4314.697412] LustreError: 141962:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 2 previous similar messages [ 4314.979281] Lustre: server umount lustre-MDT0001 complete [ 4324.391692] Lustre: DEBUG MARKER: oleg322-server.virtnet: executing unload_modules_local [ 4326.913508] Key type lgssc unregistered [ 4327.165664] LNet: 145920:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 4327.172709] LNetError: 145920:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 4327.183868] LNet: Removed LNI 192.168.203.122@tcp [ 4327.944713] Key type .llcrypt unregistered [ 4327.950335] Key type ._llcrypt unregistered [ 4355.536382] Key type ._llcrypt registered [ 4355.538528] Key type .llcrypt registered [ 4355.624312] Lustre: DEBUG MARKER: oleg322-server.virtnet: executing load_modules_local [ 4357.191750] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 4357.236320] alg: No test for adler32 (adler32-zlib) [ 4358.222514] Lustre: Lustre: Build Version: 2.17.57_1_g9da61ce [ 4358.504533] LNet: Added LNI 192.168.203.122@tcp [8/256/0/180] [ 4360.200361] Key type lgssc registered [ 4361.363314] Lustre: Echo OBD driver; http://www.lustre.org/ [ 4369.427954] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 4369.474059] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 4370.826517] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 4375.249453] Lustre: DEBUG MARKER: oleg322-server.virtnet: executing set_default_debug -1 all [ 4381.164880] LustreError: 147336:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 4381.188828] LustreError: 147336:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 2 previous similar messages [ 4383.471493] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 4383.866198] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 4388.573696] Lustre: DEBUG MARKER: oleg322-server.virtnet: executing set_default_debug -1 all [ 4395.626670] Lustre: DEBUG MARKER: oleg322-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 4399.891213] Lustre: DEBUG MARKER: oleg322-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 4405.842026] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 4406.152267] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 4409.267597] Lustre: lustre-OST0000: new connection from lustre-MDT0001-mdtlov (cleaning up unused objects from 0x280000400:72 to 0x280000400:129) [ 4411.564707] Lustre: DEBUG MARKER: oleg322-server.virtnet: executing set_default_debug -1 all [ 4415.408861] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:85 to 0x280000401:161) [ 4418.518775] Lustre: DEBUG MARKER: oleg322-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 4424.934178] Lustre: *** cfs_fail_loc=159, val=0*** [ 4440.433881] Lustre: lustre-MDT0000: Client 605eb722-f5c9-4a39-8a8d-90d8d5ed0756 (at 192.168.203.22@tcp) reconnecting [ 4446.183454] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4446.184346] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 4446.197712] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 4446.201604] Lustre: Skipped 1 previous similar message [ 4446.243403] LustreError: Skipped 1 previous similar message [ 4451.307396] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 4451.323447] Lustre: Skipped 2 previous similar messages [ 4452.384549] Lustre: server umount lustre-OST0000 complete [ 4456.420653] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4456.423881] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 4456.433448] Lustre: Skipped 1 previous similar message [ 4461.538907] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 4461.547670] Lustre: Skipped 2 previous similar messages [ 4462.526996] Lustre: server umount lustre-MDT0000 complete [ 4465.887350] LustreError: 147317:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1787121854 with bad export cookie 3416005812908138730 [ 4465.890488] LustreError: MGC192.168.203.122@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 4465.904351] LustreError: 147317:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 3 previous similar messages [ 4466.115571] Lustre: server umount lustre-MDT0001 complete [ 4476.340318] Lustre: DEBUG MARKER: oleg322-server.virtnet: executing unload_modules_local [ 4478.834767] Key type lgssc unregistered [ 4479.152826] LNet: 151006:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 4479.165413] LNetError: 151006:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 4479.200321] LNet: Removed LNI 192.168.203.122@tcp [ 4480.032239] Key type .llcrypt unregistered [ 4480.034462] Key type ._llcrypt unregistered [ 4503.781711] Key type ._llcrypt registered [ 4503.787275] Key type .llcrypt registered [ 4503.951097] Lustre: DEBUG MARKER: oleg322-server.virtnet: executing load_modules_local [ 4505.405645] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 4505.444712] alg: No test for adler32 (adler32-zlib) [ 4506.588058] Lustre: Lustre: Build Version: 2.17.57_1_g9da61ce [ 4506.949869] LNet: Added LNI 192.168.203.122@tcp [8/256/0/180] [ 4508.624144] Key type lgssc registered [ 4509.821731] Lustre: Echo OBD driver; http://www.lustre.org/ [ 4516.671822] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 4516.691159] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 4518.088301] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 4522.276731] Lustre: DEBUG MARKER: oleg322-server.virtnet: executing set_default_debug -1 all [ 4527.589810] LustreError: 152422:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 4527.612725] LustreError: 152422:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 2 previous similar messages [ 4529.550967] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 4529.934629] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 4533.722912] Lustre: DEBUG MARKER: oleg322-server.virtnet: executing set_default_debug -1 all [ 4540.576274] Lustre: DEBUG MARKER: oleg322-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 4546.011456] Lustre: DEBUG MARKER: oleg322-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 4552.124565] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 4552.491184] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 4554.558628] Lustre: lustre-OST0000: new connection from lustre-MDT0001-mdtlov (cleaning up unused objects from 0x280000400:72 to 0x280000400:161) [ 4558.320820] Lustre: DEBUG MARKER: oleg322-server.virtnet: executing set_default_debug -1 all [ 4562.765279] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:170 to 0x280000401:193) [ 4566.050859] Lustre: DEBUG MARKER: oleg322-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 4584.623207] Lustre: lustre-OST0000: evicting 30ad6772-8f65-4438-aca7-1d4ecac363b6 (at 192.168.203.22@tcp) by administrative request [ 4584.627912] Lustre: Skipped 1 previous similar message [ 4614.117658] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4614.144217] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 4614.156930] Lustre: Skipped 1 previous similar message [ 4618.789773] Lustre: server umount lustre-OST0000 complete [ 4624.354442] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4624.360364] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 4624.365855] Lustre: Skipped 2 previous similar messages [ 4628.687524] Lustre: server umount lustre-MDT0000 complete [ 4629.475376] LustreError: 155332:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 4629.491105] LustreError: 155332:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 1 previous similar message [ 4631.628715] LustreError: 152402:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1787122020 with bad export cookie 6681747287532918407 [ 4631.634869] LustreError: MGC192.168.203.122@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 4631.636403] LustreError: 152402:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 3 previous similar messages [ 4631.837424] Lustre: server umount lustre-MDT0001 complete [ 4640.733659] Lustre: DEBUG MARKER: oleg322-server.virtnet: executing unload_modules_local [ 4642.715316] Key type lgssc unregistered [ 4642.945833] LNet: 156492:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 4642.959137] LNetError: 156492:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 4642.975703] LNet: Removed LNI 192.168.203.122@tcp [ 4643.569565] Key type .llcrypt unregistered [ 4643.572932] Key type ._llcrypt unregistered [ 4715.708286] Key type ._llcrypt registered [ 4715.713131] Key type .llcrypt registered [ 4715.881502] Lustre: DEBUG MARKER: oleg322-server.virtnet: executing load_modules_local [ 4716.831575] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 4716.860942] alg: No test for adler32 (adler32-zlib) [ 4718.042470] Lustre: Lustre: Build Version: 2.17.57_1_g9da61ce [ 4718.260442] LNet: Added LNI 192.168.203.122@tcp [8/256/0/180] [ 4719.936837] Key type lgssc registered [ 4721.031187] Lustre: Echo OBD driver; http://www.lustre.org/ [ 4727.869216] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 4727.901663] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 4729.492811] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 4733.829081] Lustre: DEBUG MARKER: oleg322-server.virtnet: executing set_default_debug -1 all [ 4740.067158] LustreError: 158986:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 4740.098342] LustreError: 158986:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 2 previous similar messages [ 4741.703183] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 4742.107785] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 4746.099543] Lustre: DEBUG MARKER: oleg322-server.virtnet: executing set_default_debug -1 all [ 4752.461277] Lustre: DEBUG MARKER: oleg322-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 4757.572880] Lustre: DEBUG MARKER: oleg322-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 4763.320549] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 4763.625589] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 4766.702443] Lustre: lustre-OST0000: new connection from lustre-MDT0001-mdtlov (cleaning up unused objects from 0x280000400:72 to 0x280000400:193) [ 4768.347793] Lustre: DEBUG MARKER: oleg322-server.virtnet: executing set_default_debug -1 all [ 4774.900645] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:195 to 0x280000401:225) [ 4775.107363] Lustre: DEBUG MARKER: oleg322-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 4785.121601] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 4785.124481] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4785.135121] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 4785.147193] Lustre: Skipped 1 previous similar message [ 4789.753053] Lustre: server umount lustre-OST0000 complete [ 4793.317247] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 4793.326801] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4793.341324] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 4793.352196] Lustre: Skipped 1 previous similar message [ 4795.362183] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 4795.365357] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4799.377738] Lustre: server umount lustre-MDT0000 complete [ 4800.485708] LustreError: 158985:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 4800.504794] LustreError: 158985:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 1 previous similar message [ 4802.814126] LustreError: 158964:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1787122191 with bad export cookie 6035688156725831918 [ 4802.819795] LustreError: MGC192.168.203.122@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 4802.822684] LustreError: 158964:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 1 previous similar message [ 4803.076605] Lustre: server umount lustre-MDT0001 complete [ 4813.317509] Lustre: DEBUG MARKER: oleg322-server.virtnet: executing unload_modules_local [ 4815.780517] Key type lgssc unregistered [ 4815.998898] LNet: 162530:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 4816.009640] LNetError: 162530:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 4816.020687] LNet: Removed LNI 192.168.203.122@tcp [ 4816.801435] Key type .llcrypt unregistered [ 4816.804838] Key type ._llcrypt unregistered [ 4839.278750] Key type ._llcrypt registered [ 4839.281663] Key type .llcrypt registered [ 4839.361783] Lustre: DEBUG MARKER: oleg322-server.virtnet: executing load_modules_local [ 4840.206438] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 4840.242452] alg: No test for adler32 (adler32-zlib) [ 4841.187708] Lustre: Lustre: Build Version: 2.17.57_1_g9da61ce [ 4841.382979] LNet: Added LNI 192.168.203.122@tcp [8/256/0/180] [ 4843.016279] Key type lgssc registered [ 4844.030884] Lustre: Echo OBD driver; http://www.lustre.org/ [ 4857.220525] Lustre: DEBUG MARKER: == conf-sanity test 100: check lshowmount lists MGS, MDT, OST and 0@lo ========================================================== 02:50:43 (1787122243) [ 4871.985653] Lustre: DEBUG MARKER: oleg322-server.virtnet: executing set_hostid [ 4880.926843] Lustre: DEBUG MARKER: oleg322-server.virtnet: executing load_modules_local [ 4916.444991] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 4916.495199] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 4917.688942] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 4917.714764] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 4917.774544] Lustre: lustre-MDT0000: new disk, initializing [ 4917.856758] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 4917.868876] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 4921.720701] Lustre: DEBUG MARKER: oleg322-server.virtnet: executing set_default_debug -1 all [ 4933.597124] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 4933.729469] Lustre: 166793:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 4933.773640] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [ 4933.777357] Lustre: Skipped 1 previous similar message [ 4933.827327] Lustre: lustre-MDT0001: new disk, initializing [ 4933.892099] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 4933.952228] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [ 4933.968436] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 4938.322122] Lustre: DEBUG MARKER: oleg322-server.virtnet: executing set_default_debug -1 all [ 4948.445218] Lustre: DEBUG MARKER: oleg322-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 4954.844973] Lustre: DEBUG MARKER: oleg322-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 4962.363719] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 4962.699368] Lustre: lustre-OST0000: new disk, initializing [ 4962.704303] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 4962.714516] Lustre: 168762:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 4962.813641] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 4968.997856] Lustre: DEBUG MARKER: oleg322-server.virtnet: executing set_default_debug -1 all [ 4971.090912] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [ 4971.094693] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 4971.162734] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 4979.191401] Lustre: DEBUG MARKER: oleg322-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 4991.983389] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4991.984557] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 4992.001318] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 4992.022171] Lustre: Skipped 1 previous similar message [ 4994.409978] Lustre: server umount lustre-OST0000 complete [ 5000.675955] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 5000.682503] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5000.693623] Lustre: Skipped 1 previous similar message [ 5000.704390] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 5002.213143] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 5002.215573] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5004.321791] Lustre: server umount lustre-MDT0000 complete [ 5007.353055] LustreError: 166785:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1787122396 with bad export cookie 17335872488751588046 [ 5007.355462] LustreError: MGC192.168.203.122@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 5007.374357] LustreError: 166785:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 3 previous similar messages [ 5007.760612] Lustre: server umount lustre-MDT0001 complete [ 5018.040682] Lustre: DEBUG MARKER: oleg322-server.virtnet: executing unload_modules_local [ 5020.462658] Key type lgssc unregistered [ 5020.794585] LNet: 170670:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 5020.808623] LNetError: 170670:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 5021.863667] LNet: Removed LNI 192.168.203.122@tcp [ 5022.599102] Key type .llcrypt unregistered [ 5022.603067] Key type ._llcrypt unregistered [ 5043.826788] Key type ._llcrypt registered [ 5043.829958] Key type .llcrypt registered [ 5043.926431] Lustre: DEBUG MARKER: oleg322-server.virtnet: executing load_modules_local [ 5044.838672] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 5044.855132] alg: No test for adler32 (adler32-zlib) [ 5045.892985] Lustre: Lustre: Build Version: 2.17.57_1_g9da61ce [ 5046.118278] LNet: Added LNI 192.168.203.122@tcp [8/256/0/180] [ 5047.776184] Key type lgssc registered [ 5048.686602] Lustre: Echo OBD driver; http://www.lustre.org/ [ 5055.334944] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 5055.349161] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 5056.874498] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 5060.830756] Lustre: DEBUG MARKER: oleg322-server.virtnet: executing set_default_debug -1 all [ 5067.236571] LustreError: 172085:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 5067.263599] LustreError: 172085:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 2 previous similar messages [ 5070.506081] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 5070.800597] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 5075.372139] Lustre: DEBUG MARKER: oleg322-server.virtnet: executing set_default_debug -1 all [ 5084.110599] Lustre: DEBUG MARKER: oleg322-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 5091.130364] Lustre: DEBUG MARKER: oleg322-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 5097.837329] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 5098.292585] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 5101.375369] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:3 to 0x280000401:33) [ 5105.114549] Lustre: DEBUG MARKER: oleg322-server.virtnet: executing set_default_debug -1 all [ 5112.987341] Lustre: DEBUG MARKER: oleg322-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 5123.876583] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 5123.896452] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5123.906171] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 5123.913134] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5123.927381] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 5125.278375] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 5125.334422] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5125.368587] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 5125.393195] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5125.427522] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 5126.584388] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 5126.630335] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5126.643251] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 5126.648698] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5126.655491] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 5127.837851] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5128.998746] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 5129.019222] Lustre: Skipped 1 previous similar message [ 5129.022989] LustreError: 172105:0:(osp_precreate.c:655:osp_precreate_send()) lustre-OST0000-osc-MDT0000: can't precreate: rc = -5 [ 5129.036599] LustreError: 172105:0:(osp_precreate.c:1359:osp_precreate_thread()) lustre-OST0000-osc-MDT0000: cannot precreate objects: rc = -5 [ 5129.052598] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5129.063923] Lustre: Skipped 1 previous similar message [ 5129.068159] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 5129.072606] Lustre: Skipped 1 previous similar message [ 5129.076984] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5129.089825] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 5129.100517] Lustre: Skipped 1 previous similar message [ 5130.163272] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5131.146634] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5132.224462] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5133.011027] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 5133.020515] Lustre: Skipped 3 previous similar messages [ 5133.036732] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5134.273565] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5134.283610] Lustre: Skipped 4 previous similar messages [ 5134.289485] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 5134.294466] Lustre: Skipped 4 previous similar messages [ 5134.299648] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5134.317519] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 5134.328709] Lustre: Skipped 4 previous similar messages [ 5135.594476] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5136.714335] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5138.057322] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5138.943806] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5140.096911] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5141.075453] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 5141.080048] Lustre: Skipped 6 previous similar messages [ 5141.103147] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5141.898130] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5142.810290] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5142.821908] Lustre: Skipped 7 previous similar messages [ 5142.825170] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 5142.829208] Lustre: Skipped 7 previous similar messages [ 5142.832221] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5142.846874] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 5142.859186] Lustre: Skipped 7 previous similar messages [ 5143.818537] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5144.743718] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5145.847159] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5146.677129] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5147.553989] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5148.415531] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5149.385401] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5150.218647] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5151.138162] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5151.823622] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5152.576900] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5153.494546] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5154.600429] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5155.403138] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5156.333889] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5157.073572] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5158.107789] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 5158.119425] Lustre: Skipped 18 previous similar messages [ 5158.141602] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5159.084649] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5159.098507] Lustre: Skipped 17 previous similar messages [ 5159.102812] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 5159.109093] Lustre: Skipped 17 previous similar messages [ 5159.113697] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5159.129502] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 5159.137836] Lustre: Skipped 17 previous similar messages [ 5160.190969] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5161.163587] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5162.362030] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5163.240602] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5164.378825] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5165.115899] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5166.092418] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5166.968728] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5167.954466] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5168.828413] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5169.723932] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5170.715249] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5171.836671] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5172.849380] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5173.961376] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5175.085110] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5176.128773] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5177.090243] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5177.955646] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5178.879710] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5179.884270] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5180.881408] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5181.734423] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5182.877492] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5183.867037] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5184.810201] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5185.714751] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5186.583359] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5187.488686] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5188.501085] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5189.565775] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5190.417733] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 5190.426599] Lustre: Skipped 32 previous similar messages [ 5190.448452] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5191.340730] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5191.348835] Lustre: Skipped 32 previous similar messages [ 5191.352986] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 5191.358164] Lustre: Skipped 32 previous similar messages [ 5191.362295] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5191.374628] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 5191.377530] Lustre: Skipped 32 previous similar messages [ 5192.138419] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5193.051136] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5193.739408] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5194.646817] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5195.573956] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5196.429267] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5197.358645] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5198.345512] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5199.234748] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5200.267461] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5201.084121] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5202.080448] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5202.835402] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5203.742359] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5204.647082] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5205.572202] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5206.390490] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5207.360458] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5208.421952] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5209.578944] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5210.522878] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5211.443605] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5212.330275] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5213.301778] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5214.220267] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5215.264413] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5216.288354] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5217.322516] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5218.266563] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5219.473857] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5220.614437] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5221.649758] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5222.463483] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5223.262944] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5224.045168] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5224.989323] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5225.889606] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5226.769150] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5227.658601] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5228.407912] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5229.273830] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5230.197621] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5231.059589] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5231.779723] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5232.557462] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5233.430208] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5234.249558] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5235.077913] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5235.831975] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5236.765745] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5237.462693] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5238.299644] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5239.043489] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5239.811532] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5240.509279] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5241.463789] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5242.288864] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5243.132906] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5243.944109] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5244.846662] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5245.722838] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5246.568331] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5247.493445] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5248.263671] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5249.058608] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5249.824211] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5250.634797] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5251.527790] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5252.521244] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5253.449682] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5254.561261] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 5254.569281] Lustre: Skipped 71 previous similar messages [ 5254.605299] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5255.564442] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5255.572130] Lustre: Skipped 71 previous similar messages [ 5255.576345] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 5255.580342] Lustre: Skipped 71 previous similar messages [ 5255.583451] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5255.592655] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 5255.598120] Lustre: Skipped 71 previous similar messages [ 5256.514637] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5257.379196] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5258.235748] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5258.976502] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5259.852478] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5260.689029] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5261.525739] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5262.362781] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5263.070556] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5263.885993] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5264.760673] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5265.520511] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5266.430870] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5267.170181] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5268.295126] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5269.255395] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5270.049690] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5271.169904] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5272.102990] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5273.466435] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5274.658539] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5275.851910] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5276.906683] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5277.844464] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5279.063227] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5280.012909] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5281.353820] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5282.596824] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5283.648417] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5284.672662] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5285.579115] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5286.521533] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5287.497634] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5288.401621] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5289.597649] LustreError: 172105:0:(osp_precreate.c:655:osp_precreate_send()) lustre-OST0000-osc-MDT0000: can't precreate: rc = -5 [ 5289.623647] LustreError: 172105:0:(osp_precreate.c:1359:osp_precreate_thread()) lustre-OST0000-osc-MDT0000: cannot precreate objects: rc = -5 [ 5289.631264] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5290.537235] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5291.376456] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5292.072606] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5292.820978] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5293.746803] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5294.681587] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5295.645673] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5296.540238] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5297.320909] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5298.293188] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5299.406732] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5300.454616] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5301.555476] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5302.522209] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5303.596670] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5304.610464] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5305.424562] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5306.330954] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5307.271116] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5308.235536] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5309.176907] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5310.175366] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5311.114706] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5312.315336] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5313.457692] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5314.511106] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5315.462438] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5316.320880] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5317.175788] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5318.104841] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5318.987524] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5319.912583] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5320.913567] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5321.942898] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5322.875333] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5323.829881] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5324.745840] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5325.593421] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5326.350944] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5327.356360] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5328.187407] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5329.130904] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5329.951572] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5330.917900] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5331.930966] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5332.943129] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5333.939720] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5334.775448] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5335.590716] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5336.584706] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5337.404950] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5338.227735] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5339.029627] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5339.910848] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5340.743480] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5341.553591] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5342.276647] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5343.103591] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5343.919176] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5344.839530] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5345.613621] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5346.633418] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5347.510521] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5348.503987] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5349.405962] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5350.374791] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5351.329581] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5352.183499] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5352.981962] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5353.765938] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5354.569244] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5355.448133] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5356.156876] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5357.025871] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5357.972927] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5358.943622] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5359.858510] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5360.746321] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5361.561865] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5362.420585] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5363.342109] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5364.467670] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5365.385917] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5366.318940] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5367.158589] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5368.158621] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5369.021319] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5369.987969] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5370.892891] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5371.896729] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5372.830904] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5373.673944] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5374.399932] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5375.414956] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5376.472756] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5377.371165] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5378.486585] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5379.488708] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5380.617618] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5381.724701] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5382.625077] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 5382.635986] Lustre: Skipped 136 previous similar messages [ 5382.648192] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5383.609849] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5383.617190] Lustre: Skipped 136 previous similar messages [ 5383.627364] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 5383.637680] Lustre: Skipped 136 previous similar messages [ 5383.656679] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5383.676964] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 5383.688496] Lustre: Skipped 136 previous similar messages [ 5384.598498] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5385.523593] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5386.551270] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5387.484627] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5388.601394] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5389.648985] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5390.742460] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5391.801900] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5392.802862] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5393.809310] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5394.863976] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5395.825633] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5396.884604] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5397.784429] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5398.699530] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5399.695257] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5400.582399] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5401.506553] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5402.341740] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5403.353528] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5404.333047] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5405.284142] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5406.296484] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5407.387078] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5408.196672] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5409.177752] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5410.197194] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5411.222596] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5412.383354] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5413.435593] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5414.465759] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5415.493508] LustreError: 172105:0:(osp_precreate.c:655:osp_precreate_send()) lustre-OST0000-osc-MDT0000: can't precreate: rc = -5 [ 5415.515643] LustreError: 172105:0:(osp_precreate.c:1359:osp_precreate_thread()) lustre-OST0000-osc-MDT0000: cannot precreate objects: rc = -5 [ 5415.524952] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5416.464631] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5417.407392] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5418.550705] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5419.765329] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5420.785191] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5421.840960] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5422.808844] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5423.748239] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5424.930870] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5425.981425] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5427.191783] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5427.967308] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5429.338243] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5430.349865] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5432.138427] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5433.318837] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5434.591526] LustreError: 172105:0:(osp_precreate.c:655:osp_precreate_send()) lustre-OST0000-osc-MDT0000: can't precreate: rc = -5 [ 5434.616418] LustreError: 172105:0:(osp_precreate.c:1359:osp_precreate_thread()) lustre-OST0000-osc-MDT0000: cannot precreate objects: rc = -5 [ 5434.628611] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5435.528564] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5436.518760] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5437.882564] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5439.450972] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5440.343903] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5441.555065] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5442.908853] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5444.470991] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5446.016521] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5447.473965] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5448.967637] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5450.369043] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5451.483995] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5452.289678] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5453.369895] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5454.227256] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5455.118586] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5455.908296] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5456.972221] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5457.685239] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5458.589025] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5459.514960] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5460.557764] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5461.346038] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5462.188810] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5462.909132] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5464.038559] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5464.780329] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5465.677447] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5466.415226] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5467.273582] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5467.874171] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5468.731866] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5469.320212] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5470.258726] LustreError: 172105:0:(osp_precreate.c:655:osp_precreate_send()) lustre-OST0000-osc-MDT0000: can't precreate: rc = -5 [ 5470.265824] LustreError: 172105:0:(osp_precreate.c:1359:osp_precreate_thread()) lustre-OST0000-osc-MDT0000: cannot precreate objects: rc = -5 [ 5470.291882] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5471.150783] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5472.041231] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5472.832896] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5474.047983] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5475.041068] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5475.981582] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5476.792371] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5477.682068] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5478.518906] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5479.588683] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5480.575612] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5481.528983] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5482.251228] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5483.110659] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5484.071324] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5484.946419] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5485.809127] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5486.617944] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5487.436569] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5488.271697] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5489.032380] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5489.844029] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5490.660730] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5491.530545] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5492.308927] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5493.079645] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5493.862212] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5494.574786] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5495.287630] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5496.094791] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5496.778078] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5497.790284] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5498.757214] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5499.805120] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5500.665329] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5501.561938] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5502.404074] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5503.324126] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5504.160829] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5505.034535] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5505.806715] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5506.787054] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5507.723894] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5508.556861] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5509.439412] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5510.277611] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5511.055713] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5511.895698] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5512.812741] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5513.639674] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5514.591940] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5515.643805] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5516.513753] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5517.367935] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5518.229083] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5519.130474] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5519.912296] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5520.661416] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5521.658234] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5522.548412] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5523.482734] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5524.400052] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5525.400574] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5526.261162] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5527.192400] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5528.365542] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5529.251307] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5530.317603] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5531.218318] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5532.366582] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5533.525479] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5534.655959] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5535.537996] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5536.617968] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5537.555477] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5538.786367] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5539.800107] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5540.994792] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5542.130109] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5542.989723] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5544.239924] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5545.718874] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5546.850792] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5547.669540] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5548.764822] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5549.881853] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5551.159879] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5552.298057] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5553.719831] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5554.818762] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5556.075193] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5557.396795] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5558.521566] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5559.967188] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5561.194055] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5562.373353] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5563.630292] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5564.672380] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5566.042635] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5567.049760] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5568.015036] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5568.961174] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5569.816954] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5570.899181] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5571.987388] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5573.099419] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5574.060227] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5575.183278] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5576.440596] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5577.425671] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5578.471030] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5579.470639] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5580.583253] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5581.462078] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5582.575647] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5583.639813] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5584.574071] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5585.525345] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5586.695322] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5587.544752] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5588.515903] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5589.474202] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5590.353510] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5591.175424] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5592.169753] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5592.839111] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5593.856516] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5595.055805] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5596.077347] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5597.114732] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5597.880855] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5598.931408] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5599.934724] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5600.897022] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5601.753856] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5602.657635] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5603.462791] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5604.417611] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5605.482561] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5606.450951] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5607.497531] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5608.425909] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5609.687547] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5610.558028] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5611.432554] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5612.781030] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5613.739226] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5614.678149] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5615.488399] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5616.503334] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5617.484932] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5618.420846] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5619.169401] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5620.036949] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5620.739539] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5621.530018] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5622.364284] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5623.346566] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5624.207673] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5625.086702] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5625.808427] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5626.463969] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5627.214227] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5628.055851] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5628.889634] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5629.646052] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5630.511555] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5631.336251] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5632.262209] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5633.308374] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5634.211348] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5635.292652] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5636.341784] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5637.195704] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5638.013176] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5638.846095] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 5638.851796] Lustre: Skipped 260 previous similar messages [ 5638.873465] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5639.617798] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5639.626808] Lustre: Skipped 260 previous similar messages [ 5639.631525] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 5639.636376] Lustre: Skipped 260 previous similar messages [ 5639.640781] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5640.370832] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5640.393288] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 5640.397938] Lustre: Skipped 261 previous similar messages [ 5641.225801] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5641.914781] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5642.943222] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5643.784640] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5644.837421] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5645.658552] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5646.474909] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5647.222148] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5647.978747] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5648.764664] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5649.624589] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5650.495259] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5651.150183] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5651.851840] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5652.492255] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5653.231168] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5653.977482] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5654.884286] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5655.562791] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5656.310459] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5657.210334] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5658.152960] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5659.179660] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5659.920670] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5660.773086] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5661.618589] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5662.506605] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5663.218960] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5664.148826] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5664.931978] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5665.822794] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5666.599022] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5667.459145] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5668.237693] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5669.030873] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5669.807301] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5670.548330] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5671.318603] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5672.140592] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5672.969170] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5673.850934] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5674.562107] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5675.462070] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5676.247304] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5677.135738] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5677.865903] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5678.729117] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5679.607243] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5680.612573] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5681.473812] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5682.311893] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5683.130073] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5683.922641] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5684.826393] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5685.731384] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5686.614269] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5687.389469] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5688.431061] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5689.295691] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5690.079660] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5690.738282] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5691.399674] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5692.088449] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5692.964129] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5693.773667] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5694.672622] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5695.594575] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5696.549688] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5697.409586] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5698.193727] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5698.993241] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5700.026829] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5700.961479] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5701.852553] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5702.742696] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5703.738351] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5704.603417] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5705.404370] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5706.211585] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5706.978952] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5707.917211] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5708.719746] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5709.645017] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5710.718991] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5711.508258] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5712.264214] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5713.126798] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5714.044101] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5714.829272] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5715.672456] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5716.536941] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5717.375913] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5718.239216] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5719.260246] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5720.181108] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5721.197781] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5722.204621] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5723.078472] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5723.855687] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5724.696402] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5725.587725] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5726.408466] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5727.196204] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5727.978426] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5728.800759] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5729.706379] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5730.439534] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5731.278177] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5731.978797] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5732.672366] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5733.412425] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5734.053272] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5734.684450] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5735.468159] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5736.138154] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5737.019527] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5737.843399] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5738.746745] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5739.527138] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5740.494915] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5741.354650] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5742.239869] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5743.058647] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5743.965552] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5744.827325] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5745.778423] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5746.615431] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5747.506464] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5748.371296] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5749.314671] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5750.095506] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5750.928070] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5751.651666] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5752.500186] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5753.279975] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5754.235628] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5755.047726] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5755.952852] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5756.921679] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5757.967323] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5758.897232] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5759.977201] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5760.912426] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5761.829107] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5762.711760] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5763.583583] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5764.524040] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5765.532386] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5766.464626] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5767.463989] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5768.325913] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5769.308914] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5770.378210] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5771.228687] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5772.209405] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5773.134309] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5773.969970] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5774.851860] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5775.741820] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5776.717801] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5777.733794] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5778.606560] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5779.504571] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5780.381608] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5781.231793] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5782.097687] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5782.761634] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5783.545633] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5784.432994] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5785.382145] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5786.233031] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5787.250313] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5788.070339] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5789.002550] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5789.890622] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5790.803308] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5791.543420] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5792.262121] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5793.165181] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5794.045579] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5794.930056] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5795.756049] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5796.712553] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5797.420294] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5798.109267] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5798.976954] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5800.003920] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5801.011196] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5802.237029] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5803.858059] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5805.241107] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5806.510042] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5807.929852] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5808.970589] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5810.012045] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5811.037233] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5811.692771] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5812.768616] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5814.134867] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5815.255447] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5816.201330] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5817.086325] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5818.051171] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5819.266960] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5820.251410] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5820.928352] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5821.863888] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5822.741628] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5823.925625] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5824.950543] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5825.900486] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5826.842284] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5827.937989] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5829.037084] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5830.035679] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5831.229578] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5832.308026] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5833.228137] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5834.419192] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5835.575619] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5836.361392] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5837.479626] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5838.419727] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5839.415930] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5840.315867] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5841.246468] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5842.122406] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5843.052201] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5843.930979] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5844.861978] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5845.731948] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5846.738914] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5847.703392] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5848.710957] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5849.680580] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5850.595930] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5851.438755] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5852.417763] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5853.347746] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5854.093936] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5855.081287] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5856.062314] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5856.911263] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5857.769873] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5858.582685] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5859.564169] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5860.315140] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5861.205756] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5861.896529] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5862.629670] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5863.508775] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5864.395510] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5865.271848] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5866.250063] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5867.080854] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5867.995975] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5868.873721] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5869.742952] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5870.478727] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5871.331334] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5872.144664] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5872.907651] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5873.697538] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5874.686654] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5875.537764] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5876.264904] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5877.106769] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5878.127821] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5879.260407] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5880.328260] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5881.369038] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5882.511671] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5883.868699] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5885.145111] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5886.103277] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5886.964446] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5888.001249] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5889.290677] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5890.263744] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5891.622677] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5893.201923] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5894.468471] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5895.256612] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5896.205263] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5897.106930] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5897.987654] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5898.924187] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5899.921036] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5900.727916] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5901.583396] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5902.324886] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5903.129951] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5903.954333] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5904.686065] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5905.412969] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5906.269684] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5907.204388] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5908.038937] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5908.826208] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5909.947498] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5911.122705] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5912.146167] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5913.082839] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5913.947746] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5914.847422] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5915.850430] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5916.746235] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5917.826453] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5918.568763] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5919.458363] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5920.350896] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5921.389368] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5922.271178] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5923.313374] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5924.240847] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5925.101712] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5926.068976] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5926.998920] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5928.030740] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5929.322623] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5930.385242] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5931.372468] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5932.572977] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5933.610852] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5934.651337] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5935.881570] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5936.983858] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5938.152397] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5939.421159] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5940.358701] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5941.233736] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5942.123209] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5943.139132] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5944.142376] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5945.376149] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5946.306586] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5947.085591] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5947.917649] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5948.919526] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5949.839636] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5950.787846] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5951.710628] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5952.566341] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5953.374211] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5954.464102] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5955.673807] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5956.509598] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5957.604870] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5958.471352] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5959.335827] LustreError: 172105:0:(osp_precreate.c:655:osp_precreate_send()) lustre-OST0000-osc-MDT0000: can't precreate: rc = -5 [ 5959.350495] LustreError: 172105:0:(osp_precreate.c:1359:osp_precreate_thread()) lustre-OST0000-osc-MDT0000: cannot precreate objects: rc = -5 [ 5959.359627] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5960.222974] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5961.255111] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5962.152391] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5963.126567] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5963.984435] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5965.040161] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5966.084263] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5967.031387] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5968.022282] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5968.940323] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5969.983568] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5971.107428] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5972.011584] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5973.113539] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5974.195047] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5975.061767] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5976.151079] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5977.105926] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5978.117462] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5979.176952] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5980.070574] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5981.225193] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5982.292765] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5983.251584] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5984.348162] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5985.365729] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5986.567070] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5987.489128] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5988.499574] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5989.457697] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5990.630802] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5991.618794] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5992.802885] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5993.905377] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5994.905933] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5995.808414] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5996.705561] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5997.871073] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5998.964854] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5999.875207] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6000.781835] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6001.924673] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6003.018773] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6003.942692] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6004.953399] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6005.972599] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6007.103370] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6008.001583] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6009.025393] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6009.924246] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6010.926672] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6012.012902] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6013.109689] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6014.099604] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6015.222550] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6016.175109] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6017.335524] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6018.370646] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6019.325769] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6020.313280] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6021.361449] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6022.395352] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6023.405461] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6024.322319] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6025.289983] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6026.144818] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6027.265116] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6028.020260] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6029.019222] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6029.976701] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6030.941446] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6031.877689] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6032.885266] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6033.800979] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6034.639903] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6035.606958] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6036.708219] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6037.842044] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6038.853819] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6039.882038] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6040.859916] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6041.807258] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6042.807263] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6043.916977] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6044.878422] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6045.953279] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6046.920308] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6047.995631] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6048.939965] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6050.002823] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6050.837682] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6051.987994] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6052.934473] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6053.891131] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6054.725230] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6055.671613] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6056.738403] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6057.634286] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6058.609984] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6059.610769] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6060.620625] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6061.566316] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6062.521690] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6063.534146] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6064.432575] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6065.385110] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6066.297310] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6067.326067] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6068.468254] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6069.601399] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6511.228614] LustreError: 175300:0:(ofd_dev.c:1780:ofd_create_hdl()) lustre-OST0000: unable to precreate: rc = -28 [ 6511.236795] LustreError: 172105:0:(osp_precreate.c:655:osp_precreate_send()) lustre-OST0000-osc-MDT0000: can't precreate: rc = -28 [ 7266.273134] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 7266.284074] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 7266.310382] Lustre: Skipped 461 previous similar messages [ 7266.317833] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 7267.303446] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 7267.322257] Lustre: Skipped 1 previous similar message [ 7271.323096] Lustre: server umount lustre-OST0000 complete [ 7277.544300] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 7277.545601] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 7277.563375] Lustre: Skipped 1 previous similar message [ 7282.657770] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 7282.669171] Lustre: Skipped 1 previous similar message [ 7287.778520] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 7287.785385] Lustre: Skipped 1 previous similar message [ 7289.312141] Lustre: lustre-MDT0000 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 7289.614894] Lustre: server umount lustre-MDT0000 complete [ 7292.899038] LustreError: 172085:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 7292.922081] LustreError: 172085:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 1 previous similar message [ 7293.275605] LustreError: 172066:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1787124682 with bad export cookie 11855605538016859383 [ 7293.284589] LustreError: MGC192.168.203.122@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 7293.292339] LustreError: 172066:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 2 previous similar messages [ 7293.729287] Lustre: server umount lustre-MDT0001 complete [ 7305.586924] Lustre: DEBUG MARKER: oleg322-server.virtnet: executing unload_modules_local [ 7307.908994] Key type lgssc unregistered [ 7308.243892] LNet: 227375:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 7308.249504] LNetError: 227375:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 7308.263219] LNet: Removed LNI 192.168.203.122@tcp [ 7309.021252] Key type .llcrypt unregistered [ 7309.025962] Key type ._llcrypt unregistered [ 7339.133743] Key type ._llcrypt registered [ 7339.138687] Key type .llcrypt registered [ 7339.279903] Lustre: DEBUG MARKER: oleg322-server.virtnet: executing load_modules_local [ 7339.894631] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 7339.919420] alg: No test for adler32 (adler32-zlib) [ 7340.957964] Lustre: Lustre: Build Version: 2.17.57_1_g9da61ce [ 7341.146077] LNet: Added LNI 192.168.203.122@tcp [8/256/0/180] [ 7342.849202] Key type lgssc registered [ 7343.961273] Lustre: Echo OBD driver; http://www.lustre.org/ [ 7353.475577] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 7353.494739] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 7354.951557] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 7360.176641] Lustre: DEBUG MARKER: oleg322-server.virtnet: executing set_default_debug -1 all [ 7365.105884] LustreError: 228791:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 7365.148530] LustreError: 228791:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 2 previous similar messages [ 7370.180493] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 7370.212106] LustreError: 228792:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 7370.651758] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 7375.180381] Lustre: DEBUG MARKER: oleg322-server.virtnet: executing set_default_debug -1 all [ 7383.173724] Lustre: DEBUG MARKER: oleg322-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 7389.017422] Lustre: DEBUG MARKER: oleg322-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 7395.139161] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 7395.550300] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 7399.680056] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:49665 to 0x280000401:49696) [ 7401.342654] Lustre: DEBUG MARKER: oleg322-server.virtnet: executing set_default_debug -1 all [ 7408.674933] Lustre: DEBUG MARKER: oleg322-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 7414.473060] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 7416.349868] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 7416.360167] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 7416.366870] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7416.374971] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 7416.377471] LustreError: 228811:0:(osp_precreate.c:1268:osp_precreate_thread()) cfs_fail_timeout id 2107 sleeping for 20000ms [ 7420.402213] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 7420.417586] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 7420.439403] Lustre: Skipped 1 previous similar message [ 7421.788782] Lustre: lustre-OST0000: Not available for connect from 192.168.203.22@tcp (stopping) [ 7421.796874] Lustre: Skipped 1 previous similar message [ 7425.063818] Lustre: server umount lustre-OST0000 complete [ 7436.456141] LustreError: 228811:0:(osp_precreate.c:1268:osp_precreate_thread()) cfs_fail_timeout id 2107 awake [ 7459.361688] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 7459.712590] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 7460.717766] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7460.732959] LustreError: lustre-OST0000-osc-MDT0001: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7460.734745] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 7460.752949] Lustre: Skipped 1 previous similar message [ 7466.589872] Lustre: DEBUG MARKER: oleg322-server.virtnet: executing set_default_debug -1 all [ 7473.796423] Lustre: DEBUG MARKER: oleg322-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 7475.496570] Lustre: DEBUG MARKER: osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid in FULL state after 0 sec [ 7480.750583] Lustre: DEBUG MARKER: oleg322-client.virtnet: executing wait_import_state (FULL|IDLE) osc.lustre-OST0000-osc-ffff933585517800.ost_server_uuid 50 [ 7482.217894] Lustre: DEBUG MARKER: osc.lustre-OST0000-osc-ffff933585517800.ost_server_uuid in FULL state after 0 sec [ 7487.458987] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 7487.476072] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 7487.505305] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 7491.554992] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 7491.556283] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 7491.559760] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 7491.562418] Lustre: Skipped 1 previous similar message [ 7492.742878] Lustre: server umount lustre-OST0000 complete [ 7496.673990] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 7496.681039] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 7496.694496] Lustre: Skipped 1 previous similar message [ 7502.458476] Lustre: server umount lustre-MDT0000 complete [ 7506.068923] LustreError: 228774:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1787124894 with bad export cookie 17084126117421587426 [ 7506.083455] LustreError: 228774:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 3 previous similar messages [ 7506.083625] LustreError: MGC192.168.203.122@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 7506.506342] Lustre: server umount lustre-MDT0001 complete [ 7516.115637] Lustre: DEBUG MARKER: oleg322-server.virtnet: executing unload_modules_local [ 7518.425370] Key type lgssc unregistered [ 7518.670886] LNet: 234003:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 7518.682081] LNetError: 234003:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 7518.702134] LNet: Removed LNI 192.168.203.122@tcp [ 7519.527935] Key type .llcrypt unregistered [ 7519.530137] Key type ._llcrypt unregistered [ 7552.589673] Key type ._llcrypt registered [ 7552.592028] Key type .llcrypt registered [ 7552.741164] Lustre: DEBUG MARKER: oleg322-server.virtnet: executing load_modules_local [ 7553.858682] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 7553.876814] alg: No test for adler32 (adler32-zlib) [ 7555.193480] Lustre: Lustre: Build Version: 2.17.57_1_g9da61ce [ 7555.536832] LNet: Added LNI 192.168.203.122@tcp [8/256/0/180] [ 7557.272233] Key type lgssc registered [ 7558.425659] Lustre: Echo OBD driver; http://www.lustre.org/ [ 7567.774307] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 7567.789333] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 7569.431131] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 7574.507882] Lustre: DEBUG MARKER: oleg322-server.virtnet: executing set_default_debug -1 all [ 7579.618955] LustreError: 235734:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 7579.638937] LustreError: 235734:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 2 previous similar messages [ 7584.737313] LustreError: 235735:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 7584.777243] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 7585.398278] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 7589.943290] Lustre: DEBUG MARKER: oleg322-server.virtnet: executing set_default_debug -1 all [ 7593.165806] Lustre: Modifying parameter general.debug_raw_pointers=Y in log params [ 7599.659642] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 7600.058309] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 7606.319523] Lustre: DEBUG MARKER: oleg322-server.virtnet: executing set_default_debug -1 all [ 7614.384445] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:49698 to 0x280000401:49728) [ 7615.140826] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 7615.292244] Lustre: lustre-OST0001: new disk, initializing [ 7615.298791] Lustre: srv-lustre-OST0001: No data found on store. Initialize space. [ 7615.319881] Lustre: 238040:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 7615.407270] Lustre: lustre-OST0001: Imperative Recovery not enabled, recovery window 60-180 [ 7621.457617] Lustre: DEBUG MARKER: oleg322-server.virtnet: executing set_default_debug -1 all [ 7622.271773] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x00000002c0000400-0x0000000300000400]:1:ost [ 7622.303313] Lustre: cli-lustre-OST0001-super: Allocated super-sequence [0x00000002c0000400-0x0000000300000400]:1:ost] [ 7622.410189] Lustre: lustre-OST0001-osc-MDT0000: update sequence from 0x100010000 to 0x2c0000401 [ 7632.179688] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 7643.726802] Lustre: Setting parameter general.lod.*.mdt_hash=crush in log params [ 7658.468363] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 7658.473964] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 7658.478659] Lustre: Skipped 3 previous similar messages [ 7658.489780] Lustre: Skipped 3 previous similar messages [ 7663.526587] Lustre: server umount lustre-MDT0000 complete [ 7663.589164] LustreError: 236462:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 7663.616887] LustreError: 236462:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 3 previous similar messages [ 7667.667804] LustreError: 239041:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1787125056 with bad export cookie 4475354499250586471 [ 7667.669820] LustreError: MGC192.168.203.122@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 7667.677236] LustreError: 239041:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 7668.181248] Lustre: server umount lustre-MDT0001 complete [ 7681.541442] Lustre: server umount lustre-OST0000 complete [ 7694.791176] Lustre: server umount lustre-OST0001 complete [ 7716.436774] Lustre: DEBUG MARKER: oleg322-server.virtnet: executing load_modules_local [ 7726.298144] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 7726.745928] LustreError: 241152:0:(ldlm_lib.c:1192:target_handle_connect()) mylustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 7726.810356] Lustre: mylustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 7731.279242] Lustre: DEBUG MARKER: oleg322-server.virtnet: executing set_default_debug -1 all [ 7732.194357] LustreError: 241153:0:(ldlm_lib.c:1192:target_handle_connect()) mylustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 7739.168810] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 7739.518209] Lustre: mylustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 7745.274743] Lustre: DEBUG MARKER: oleg322-server.virtnet: executing set_default_debug -1 all [ 7748.711502] Lustre: 242294:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify general/debug_raw_pointers=Y (mode = 0) failed: rc = -17 [ 7756.088080] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 7756.578145] Lustre: mylustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 7762.481955] Lustre: DEBUG MARKER: oleg322-server.virtnet: executing set_default_debug -1 all [ 7764.781656] LustreError: 243126:0:(ldlm_lib.c:1192:target_handle_connect()) mylustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 7764.794805] Lustre: mylustre-OST0000: new connection from mylustre-MDT0001-mdtlov (cleaning up unused objects from 0x280000400:100 to 0x280000400:129) [ 7764.799339] LustreError: 243126:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 1 previous similar message [ 7771.570737] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 7772.846681] Lustre: mylustre-OST0000: new connection from mylustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:49698 to 0x280000401:49760) [ 7778.071804] Lustre: DEBUG MARKER: oleg322-server.virtnet: executing set_default_debug -1 all [ 7785.421436] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 7788.668938] Lustre: 244166:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify general/lod.*.mdt_hash=crush (mode = 0) failed: rc = -17 [ 7800.802081] LustreError: mylustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 7800.813294] Lustre: mylustre-MDT0000-osp-MDT0001: Connection to mylustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 7800.826710] Lustre: mylustre-MDT0000: Not available for connect from 0@lo (stopping) [ 7803.880145] Lustre: mylustre-MDT0000: Not available for connect from 0@lo (stopping) [ 7803.889361] Lustre: mylustre-MDT0000-lwp-OST0000: Connection to mylustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 7803.901238] Lustre: Skipped 2 previous similar messages [ 7805.517176] Lustre: server umount mylustre-MDT0000 complete [ 7808.820234] LustreError: 241128:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1787125197 with bad export cookie 4475354499250589334 [ 7808.824822] LustreError: MGC192.168.203.122@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 7808.837357] LustreError: 241128:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 7809.003299] LustreError: 243478:0:(ldlm_lib.c:1192:target_handle_connect()) mylustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 7809.004743] Lustre: mylustre-MDT0001-lwp-OST0000: Connection to mylustre-MDT0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 7809.011278] LustreError: 243478:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 1 previous similar message [ 7809.126681] Lustre: server umount mylustre-MDT0001 complete [ 7818.893656] Lustre: server umount mylustre-OST0000 complete [ 7822.978552] Lustre: server umount mylustre-OST0001 complete [ 7842.774476] Lustre: DEBUG MARKER: oleg322-server.virtnet: executing load_modules_local [ 7851.921747] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 7852.309627] LustreError: 246473:0:(ldlm_lib.c:1192:target_handle_connect()) tfs-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 7852.328624] LustreError: 246473:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 3 previous similar messages [ 7852.383302] Lustre: tfs-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 7852.391583] Lustre: Skipped 1 previous similar message [ 7856.119821] Lustre: DEBUG MARKER: oleg322-server.virtnet: executing set_default_debug -1 all [ 7864.742889] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 7870.908534] Lustre: DEBUG MARKER: oleg322-server.virtnet: executing set_default_debug -1 all [ 7874.751915] Lustre: 247616:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify general/debug_raw_pointers=Y (mode = 0) failed: rc = -17 [ 7884.847381] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 7889.472083] Lustre: tfs-OST0000: new connection from tfs-MDT0001-mdtlov (cleaning up unused objects from 0x280000400:131 to 0x280000400:161) [ 7893.293801] Lustre: DEBUG MARKER: oleg322-server.virtnet: executing set_default_debug -1 all [ 7897.669315] Lustre: tfs-OST0000: new connection from tfs-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:49698 to 0x280000401:49792) [ 7903.235915] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 7910.214236] Lustre: DEBUG MARKER: oleg322-server.virtnet: executing set_default_debug -1 all [ 7917.920587] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 7921.685490] Lustre: 249489:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify general/lod.*.mdt_hash=crush (mode = 0) failed: rc = -17 [ 7929.326287] Lustre: tfs-MDT0000-osp-MDT0001: Connection to tfs-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 7929.340571] Lustre: tfs-MDT0000: Not available for connect from 0@lo (stopping) [ 7929.351977] Lustre: Skipped 3 previous similar messages [ 7929.370890] Lustre: Skipped 6 previous similar messages [ 7934.441021] Lustre: tfs-MDT0000: Not available for connect from 0@lo (stopping) [ 7934.446486] Lustre: Skipped 3 previous similar messages [ 7934.984151] Lustre: server umount tfs-MDT0000 complete [ 7938.847667] LustreError: 246449:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1787125327 with bad export cookie 4475354499250591280 [ 7938.855454] LustreError: MGC192.168.203.122@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 7938.865320] LustreError: 246449:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 7939.167584] Lustre: server umount tfs-MDT0001 complete [ 7953.326276] Lustre: server umount tfs-OST0000 complete [ 7955.424113] Lustre: 234995:0:(client.c:2490:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1787125328/real 1787125328] req@ffff8beafcffce00 x1873936325493632/t0(0) o400->tfs-MDT0001-lwp-OST0001@0@lo:12/10 lens 224/224 e 0 to 1 dl 1787125344 ref 1 fl Rpc:XNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 [ 7955.445993] Lustre: tfs-MDT0001-lwp-OST0001: Connection to tfs-MDT0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 7955.458194] Lustre: Skipped 1 previous similar message [ 7957.212217] Lustre: server umount tfs-OST0001 complete [ 7978.451759] Lustre: DEBUG MARKER: oleg322-server.virtnet: executing load_modules_local [ 7988.229229] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 7988.672927] LustreError: 251738:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 7988.694632] LustreError: 251738:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 8 previous similar messages [ 7988.743354] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 7988.748434] Lustre: Skipped 3 previous similar messages [ 7993.237806] Lustre: DEBUG MARKER: oleg322-server.virtnet: executing set_default_debug -1 all [ 8002.889749] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 8007.873897] Lustre: DEBUG MARKER: oleg322-server.virtnet: executing set_default_debug -1 all [ 8011.658834] Lustre: 252881:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify general/debug_raw_pointers=Y (mode = 0) failed: rc = -17 [ 8019.685583] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 8026.710279] Lustre: DEBUG MARKER: oleg322-server.virtnet: executing set_default_debug -1 all [ 8028.276122] Lustre: lustre-OST0000: new connection from lustre-MDT0001-mdtlov (cleaning up unused objects from 0x280000400:131 to 0x280000400:193) [ 8033.406662] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:49698 to 0x280000401:49824) [ 8035.863166] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 8042.252208] Lustre: DEBUG MARKER: oleg322-server.virtnet: executing set_default_debug -1 all [ 8050.160612] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 8054.203792] Lustre: 254751:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify general/lod.*.mdt_hash=crush (mode = 0) failed: rc = -17 [ 8061.064756] Lustre: DEBUG MARKER: == conf-sanity test 104a: Make sure user defined options are reflected in mount ========================================================== 03:44:07 (1787125447) [ 8077.288050] Lustre: lustre-MDT0000-lwp-OST0000: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 8077.290836] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 8077.296065] Lustre: Skipped 3 previous similar messages [ 8077.309225] Lustre: Skipped 3 previous similar messages [ 8083.675962] Lustre: server umount lustre-MDT0000 complete [ 8088.020332] LustreError: 251714:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1787125476 with bad export cookie 4475354499250592981 [ 8088.036195] LustreError: 251714:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 1 previous similar message [ 8088.038583] LustreError: MGC192.168.203.122@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 8093.677052] Lustre: lustre-MDT0001: Not available for connect from 0@lo (stopping) [ 8093.685152] Lustre: Skipped 6 previous similar messages [ 8102.368408] Lustre: lustre-MDT0001 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 8102.548388] Lustre: server umount lustre-MDT0001 complete [ 8112.403479] Lustre: server umount lustre-OST0000 complete [ 8122.192933] Lustre: server umount lustre-OST0001 complete [ 8128.764501] Lustre: DEBUG MARKER: oleg322-server.virtnet: executing set_hostid [ 8138.443228] Lustre: DEBUG MARKER: oleg322-server.virtnet: executing load_modules_local [ 8175.433578] LDISKFS-fs (dm-0): Mount option "noacl" will be removed by 3.5 [ 8175.433578] Contact linux-ldiskfs@vger.kernel.org if you think we should keep it. [ 8175.433578] [ 8175.467515] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: acl,user_xattr,errors=remount-ro,noacl,no_mbcache,nodelalloc [ 8175.722103] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 8175.752252] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 8175.794452] Lustre: lustre-MDT0000: new disk, initializing [ 8175.848781] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 8175.855295] Lustre: Skipped 3 previous similar messages [ 8175.867165] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 8180.085866] Lustre: DEBUG MARKER: oleg322-server.virtnet: executing set_default_debug -1 all [ 8190.653320] LDISKFS-fs (dm-1): Mount option "noacl" will be removed by 3.5 [ 8190.653320] Contact linux-ldiskfs@vger.kernel.org if you think we should keep it. [ 8190.653320] [ 8190.676795] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: acl,user_xattr,errors=remount-ro,noacl,no_mbcache,nodelalloc [ 8190.833329] Lustre: 257937:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 8190.871473] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [ 8190.874832] Lustre: Skipped 1 previous similar message [ 8190.935932] Lustre: lustre-MDT0001: new disk, initializing [ 8191.092204] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [ 8191.104979] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 8195.238292] Lustre: DEBUG MARKER: oleg322-server.virtnet: executing set_default_debug -1 all [ 8205.493482] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 8205.923375] Lustre: lustre-OST0000: new disk, initializing [ 8205.940129] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 8205.951407] Lustre: 259522:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 8207.861835] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [ 8207.878795] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 8208.025750] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 8212.410496] Lustre: DEBUG MARKER: oleg322-server.virtnet: executing set_default_debug -1 all [ 8222.688785] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 8222.843462] Lustre: lustre-OST0001: new disk, initializing [ 8222.847852] Lustre: srv-lustre-OST0001: No data found on store. Initialize space. [ 8222.855625] Lustre: 260392:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 8224.364055] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x00000002c0000400-0x0000000300000400]:1:ost [ 8224.384247] Lustre: cli-lustre-OST0001-super: Allocated super-sequence [0x00000002c0000400-0x0000000300000400]:1:ost] [ 8224.457600] Lustre: lustre-OST0001-osc-MDT0000: update sequence from 0x100010000 to 0x2c0000401 [ 8228.333092] Lustre: DEBUG MARKER: oleg322-server.virtnet: executing set_default_debug -1 all [ 8237.700547] Lustre: DEBUG MARKER: == conf-sanity test 104b: Mount uses last flock argument ========================================================== 03:47:04 (1787125624) [ 8251.467327] Lustre: DEBUG MARKER: == conf-sanity test 105: check file creation for ro and rw bind mnt pt ========================================================== 03:47:18 (1787125638) [ 8269.286056] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 8269.290217] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 8269.298787] LustreError: Skipped 1 previous similar message [ 8269.314892] Lustre: Skipped 2 previous similar messages [ 8269.320434] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 8269.331316] Lustre: Skipped 3 previous similar messages [ 8272.770305] Lustre: server umount lustre-OST0000 complete [ 8275.943882] LustreError: 259506:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-OST0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 8275.960529] LustreError: 259506:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 16 previous similar messages [ 8277.990063] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 8282.403793] Lustre: server umount lustre-MDT0000 complete [ 8285.865878] LustreError: 259521:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1787125674 with bad export cookie 4475354499250594535 [ 8285.872107] LustreError: MGC192.168.203.122@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 8285.876942] LustreError: 259521:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 2 previous similar messages [ 8292.251800] Lustre: server umount lustre-MDT0001 complete [ 8309.128665] Lustre: DEBUG MARKER: oleg322-server.virtnet: executing unload_modules_local [ 8382.934958] Lustre: server umount lustre-OST0001 complete [ 8390.679379] Lustre: DEBUG MARKER: oleg322-server.virtnet: executing set_hostid [ 8406.504993] Lustre: DEBUG MARKER: oleg322-server.virtnet: executing load_modules_local [ 8441.681185] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 8442.000941] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 8442.034492] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 8442.163205] Lustre: lustre-MDT0000: new disk, initializing [ 8442.262600] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 8442.268537] Lustre: Skipped 3 previous similar messages [ 8442.285305] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 8446.316948] Lustre: DEBUG MARKER: oleg322-server.virtnet: executing set_default_debug -1 all [ 8456.826757] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 8456.980560] Lustre: 265188:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 8457.223324] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 8461.578125] Lustre: DEBUG MARKER: oleg322-server.virtnet: executing set_default_debug -1 all [ 8473.660841] Lustre: DEBUG MARKER: oleg322-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 8479.366658] Lustre: DEBUG MARKER: oleg322-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 8487.049892] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 8487.281254] Lustre: lustre-OST0000: new disk, initializing [ 8487.284552] Lustre: Skipped 1 previous similar message [ 8487.291396] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 8487.299121] Lustre: Skipped 2 previous similar messages [ 8487.304977] Lustre: 267157:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 8489.218991] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [ 8489.231515] Lustre: Skipped 1 previous similar message [ 8489.241848] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 8489.409262] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 8494.385758] Lustre: DEBUG MARKER: oleg322-server.virtnet: executing set_default_debug -1 all [ 8504.278500] Lustre: DEBUG MARKER: oleg322-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 8515.043753] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 8515.053536] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 8515.072334] LustreError: Skipped 1 previous similar message [ 8515.073039] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 8515.073043] Lustre: Skipped 8 previous similar messages [ 8515.133516] Lustre: Skipped 5 previous similar messages [ 8518.150204] Lustre: server umount lustre-OST0000 complete [ 8523.748097] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 8529.307534] Lustre: server umount lustre-MDT0000 complete [ 8533.488838] LustreError: 265180:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1787125922 with bad export cookie 4475354499250595998 [ 8533.493950] LustreError: MGC192.168.203.122@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 8533.518628] LustreError: 265180:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 5 previous similar messages [ 8534.096298] Lustre: server umount lustre-MDT0001 complete [ 8546.929503] Lustre: DEBUG MARKER: oleg322-server.virtnet: executing unload_modules_local [ 8549.391873] Key type lgssc unregistered [ 8549.687955] LNet: 268921:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 8549.695729] LNetError: 268921:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 8549.724469] LNet: Removed LNI 192.168.203.122@tcp [ 8550.584152] Key type .llcrypt unregistered [ 8550.585896] Key type ._llcrypt unregistered [ 8577.379475] Key type ._llcrypt registered [ 8577.381415] Key type .llcrypt registered [ 8577.495530] Lustre: DEBUG MARKER: oleg322-server.virtnet: executing load_modules_local [ 8578.520926] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 8578.708324] alg: No test for adler32 (adler32-zlib) [ 8579.737230] Lustre: Lustre: Build Version: 2.17.57_1_g9da61ce [ 8580.065877] LNet: Added LNI 192.168.203.122@tcp [8/256/0/180] [ 8581.712161] Key type lgssc registered [ 8582.656734] Lustre: Echo OBD driver; http://www.lustre.org/ [ 8590.168491] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 8590.195772] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 8591.616452] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 8596.194802] Lustre: DEBUG MARKER: oleg322-server.virtnet: executing set_default_debug -1 all [ 8602.095095] LustreError: 270383:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 8602.125793] LustreError: 270383:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 2 previous similar messages [ 8605.335301] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 8605.685252] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 8610.251150] Lustre: DEBUG MARKER: oleg322-server.virtnet: executing set_default_debug -1 all [ 8617.642687] Lustre: DEBUG MARKER: oleg322-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 8623.387048] Lustre: DEBUG MARKER: oleg322-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 8629.010727] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 8629.526213] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 8636.378333] Lustre: DEBUG MARKER: oleg322-server.virtnet: executing set_default_debug -1 all [ 8636.801558] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:3 to 0x280000401:33) [ 8646.802360] Lustre: DEBUG MARKER: oleg322-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 8648.755494] Lustre: Setting parameter lustre-OST0000.ost.unknown_param=50 in log lustre-OST0000 [ 8652.258455] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 8652.263221] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 8652.282973] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 8652.296424] Lustre: Skipped 1 previous similar message [ 8652.313547] Lustre: Skipped 1 previous similar message [ 8657.380379] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 8657.390716] Lustre: Skipped 1 previous similar message [ 8658.138475] Lustre: server umount lustre-OST0000 complete [ 8667.111947] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 8667.120901] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 8667.138436] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 8669.709410] Lustre: server umount lustre-MDT0000 complete [ 8672.737396] LustreError: 270377:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 8672.752039] LustreError: 270377:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 1 previous similar message [ 8673.427635] LustreError: 270365:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1787126062 with bad export cookie 1924875593328166530 [ 8673.434920] LustreError: MGC192.168.203.122@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 8673.992789] Lustre: server umount lustre-MDT0001 complete [ 8684.844604] Lustre: DEBUG MARKER: oleg322-server.virtnet: executing unload_modules_local [ 8687.316208] Key type lgssc unregistered [ 8687.584526] LNet: 273972:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 8687.589725] LNetError: 273972:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 8687.608787] LNet: Removed LNI 192.168.203.122@tcp [ 8688.573733] Key type .llcrypt unregistered [ 8688.576113] Key type ._llcrypt unregistered [ 8705.246392] Key type ._llcrypt registered [ 8705.249490] Key type .llcrypt registered [ 8705.356501] Lustre: DEBUG MARKER: oleg322-server.virtnet: executing load_modules_local [ 8706.780589] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 8706.809342] alg: No test for adler32 (adler32-zlib) [ 8707.895168] Lustre: Lustre: Build Version: 2.17.57_1_g9da61ce [ 8708.135275] LNet: Added LNI 192.168.203.122@tcp [8/256/0/180] [ 8709.824187] Key type lgssc registered [ 8711.024322] Lustre: Echo OBD driver; http://www.lustre.org/ [ 8720.893204] Lustre: lustre-OST0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 8720.936509] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 8736.544659] LustreError: Failed to get MGS log lustre-sptlrpc and no local copy. [ 8741.664310] LustreError: 275168:0:(mgc_request.c:233:do_config_log_add()) MGC192.168.203.122@tcp: failed processing log, type 4: rc = -110 [ 8768.352700] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 8776.566592] Lustre: DEBUG MARKER: oleg322-server.virtnet: executing set_default_debug -1 all [ 8785.109216] Lustre: DEBUG MARKER: oleg322-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 8787.409696] Lustre: server umount lustre-OST0000 complete [ 8806.317382] Lustre: DEBUG MARKER: oleg322-server.virtnet: executing unload_modules_local [ 8809.186684] Key type lgssc unregistered [ 8809.482375] LNet: 276803:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 8809.513197] LNetError: 276803:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 8809.536587] LNet: Removed LNI 192.168.203.122@tcp [ 8810.621361] Key type .llcrypt unregistered [ 8810.627615] Key type ._llcrypt unregistered [ 8850.041581] Key type ._llcrypt registered [ 8850.043788] Key type .llcrypt registered [ 8850.124938] Lustre: DEBUG MARKER: oleg322-server.virtnet: executing load_modules_local [ 8851.528733] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 8851.560906] alg: No test for adler32 (adler32-zlib) [ 8852.682611] Lustre: Lustre: Build Version: 2.17.57_1_g9da61ce [ 8853.000068] LNet: Added LNI 192.168.203.122@tcp [8/256/0/180] [ 8854.696310] Key type lgssc registered [ 8856.000217] Lustre: Echo OBD driver; http://www.lustre.org/ [ 8869.011590] loop0: detected capacity change from 0 to 204800000 [ 8869.128706] blk_update_request: operation not supported error, dev loop0, sector 399872 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [ 8869.139461] blk_update_request: operation not supported error, dev loop0, sector 208 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [ 8869.546808] blk_update_request: operation not supported error, dev loop0, sector 160232 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [ 8870.630309] loop0: detected capacity change from 0 to 204800000 [ 8870.853959] blk_update_request: operation not supported error, dev loop0, sector 399872 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [ 8870.868170] blk_update_request: operation not supported error, dev loop0, sector 208 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [ 8871.163839] blk_update_request: operation not supported error, dev loop0, sector 160232 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [ 8872.032898] loop0: detected capacity change from 0 to 204800000 [ 8872.238951] blk_update_request: operation not supported error, dev loop0, sector 399872 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [ 8872.254792] blk_update_request: operation not supported error, dev loop0, sector 8224 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [ 8872.358063] blk_update_request: operation not supported error, dev loop0, sector 58264 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [ 8873.631835] loop0: detected capacity change from 0 to 204800000 [ 8873.867333] blk_update_request: operation not supported error, dev loop0, sector 399872 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [ 8875.416140] loop0: detected capacity change from 0 to 204800000 [ 8875.447534] LDISKFS-fs (loop0): mounted filesystem with ordered data mode. Opts: (null) [ 8879.979954] loop0: detected capacity change from 0 to 204800000 [ 8880.011498] LDISKFS-fs (loop0): mounted filesystem with ordered data mode. Opts: (null) [ 8884.941055] loop0: detected capacity change from 0 to 204800000 [ 8885.015706] LDISKFS-fs (loop0): mounted filesystem with ordered data mode. Opts: (null) [ 8890.463824] loop0: detected capacity change from 0 to 204800000 [ 8890.516196] LDISKFS-fs (loop0): mounted filesystem with ordered data mode. Opts: (null) [ 8895.380853] loop0: detected capacity change from 0 to 204800000 [ 8895.591642] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 8895.672022] LDISKFS-fs (loop0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 8895.789301] Lustre: 280215:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0000: restore index 'fld' with [0x200000001:0x3:0x0]: rc = 0 [ 8895.802480] Lustre: 280215:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0000: restore index 'nodemap' with [0x200000003:0x3:0x0]: rc = 0 [ 8895.834348] Lustre: 280215:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0000: restore index '0x1010000' with [0x200000003:0xb:0x0]: rc = 0 [ 8895.856603] Lustre: 280215:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0000: restore index '0x1010000-MDT0000' with [0x200000005:0x2:0x0]: rc = 0 [ 8895.881530] Lustre: 280215:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0000: restore index '0x2010000-MDT0000' with [0x200000005:0x3:0x0]: rc = 0 [ 8895.895249] Lustre: 280215:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0000: restore index '0x10000-MDT0000' with [0x200000005:0x1:0x0]: rc = 0 [ 8895.925490] Lustre: 280215:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0000: restore index '0x2010000' with [0x200000003:0xc:0x0]: rc = 0 [ 8895.946531] Lustre: 280215:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0000: restore index '0x10000' with [0x200000003:0xa:0x0]: rc = 0 [ 8895.970958] Lustre: 280215:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0000: restore index '0x1010000' with [0x200000006:0x1010000:0x0]: rc = 0 [ 8896.003257] Lustre: 280215:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0000: restore index '0x2010000' with [0x200000006:0x2010000:0x0]: rc = 0 [ 8896.030511] Lustre: 280215:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0000: restore index '0x10000' with [0x200000006:0x10000:0x0]: rc = 0 [ 8896.048514] Lustre: 280215:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0000: restore index '0x2020000' with [0x200000006:0x2020000:0x0]: rc = 0 [ 8896.066438] Lustre: 280215:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0000: restore index '0x20000' with [0x200000006:0x20000:0x0]: rc = 0 [ 8896.080600] Lustre: 280215:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0000: restore index '0x1020000' with [0x200000006:0x1020000:0x0]: rc = 0 [ 8902.135662] Lustre: server umount MGS complete [ 8903.228405] loop0: detected capacity change from 0 to 204800000 [ 8903.363989] LDISKFS-fs (loop0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 8903.960265] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 8904.095654] Lustre: lustre-MDT0000: Aborting client recovery [ 8905.246430] loop1: detected capacity change from 0 to 204800000 [ 8905.493819] LDISKFS-fs (loop1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 8905.645613] Lustre: 280649:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0001: restore index 'fld' with [0x200000001:0x3:0x0]: rc = 0 [ 8905.660648] Lustre: 280649:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0001: restore index '0x10000-MDT0001' with [0x200000005:0x1:0x0]: rc = 0 [ 8905.686678] Lustre: 280649:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0001: restore index '0x2010000' with [0x200000003:0x6:0x0]: rc = 0 [ 8905.706450] Lustre: 280649:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0001: restore index '0x2010000-MDT0001' with [0x200000005:0x3:0x0]: rc = 0 [ 8905.721646] Lustre: 280649:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0001: restore index '0x10000' with [0x200000003:0x4:0x0]: rc = 0 [ 8905.731081] Lustre: 280649:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0001: restore index '0x1010000' with [0x200000003:0x5:0x0]: rc = 0 [ 8905.753427] Lustre: 280649:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0001: restore index '0x1010000-MDT0001' with [0x200000005:0x2:0x0]: rc = 0 [ 8905.814056] Lustre: Found index 1 for lustre-MDT0001, updating log [ 8906.049944] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 8906.113866] Lustre: lustre-MDT0001: Aborting client recovery [ 8907.035843] loop2: detected capacity change from 0 to 204800000 [ 8907.262468] LDISKFS-fs (loop2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 8907.322422] Lustre: 280729:0:(osd_scrub.c:1854:osd_index_restore()) lustre-OST0000: restore index 'fld' with [0x200000001:0x3:0x0]: rc = 0 [ 8907.343768] Lustre: 280729:0:(osd_scrub.c:1854:osd_index_restore()) lustre-OST0000: restore index '0x1020000' with [0x200000003:0x6:0x0]: rc = 0 [ 8907.387444] Lustre: 280729:0:(osd_scrub.c:1854:osd_index_restore()) lustre-OST0000: restore index '0x20000' with [0x200000003:0x5:0x0]: rc = 0 [ 8907.422626] Lustre: 280729:0:(osd_scrub.c:1854:osd_index_restore()) lustre-OST0000: restore index '0x1020000-OST0000' with [0x200000005:0x2:0x0]: rc = 0 [ 8907.460881] Lustre: 280729:0:(osd_scrub.c:1854:osd_index_restore()) lustre-OST0000: restore index '0x20000-OST0000' with [0x200000005:0x1:0x0]: rc = 0 [ 8907.472645] Lustre: 280729:0:(osd_scrub.c:1854:osd_index_restore()) lustre-OST0000: restore index '0x2020000-OST0000' with [0x200000005:0x3:0x0]: rc = 0 [ 8907.499795] Lustre: 280729:0:(osd_scrub.c:1854:osd_index_restore()) lustre-OST0000: restore index '0x2020000' with [0x200000003:0x7:0x0]: rc = 0 [ 8907.679995] Lustre: Found index 0 for lustre-OST0000, updating log [ 8907.734762] Lustre: 280748:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 8907.824836] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 8907.866239] Lustre: lustre-OST0000: Aborting recovery [ 8909.240096] loop3: detected capacity change from 0 to 204800000 [ 8909.392384] LDISKFS-fs (loop3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 8909.515486] Lustre: 280814:0:(osd_scrub.c:1854:osd_index_restore()) lustre-OST0001: restore index 'fld' with [0x200000001:0x3:0x0]: rc = 0 [ 8909.569225] Lustre: 280814:0:(osd_scrub.c:1854:osd_index_restore()) lustre-OST0001: restore index '0x2020000' with [0x200000003:0x7:0x0]: rc = 0 [ 8909.600324] Lustre: 280814:0:(osd_scrub.c:1854:osd_index_restore()) lustre-OST0001: restore index '0x20000-OST0001' with [0x200000005:0x1:0x0]: rc = 0 [ 8909.614191] Lustre: 280814:0:(osd_scrub.c:1854:osd_index_restore()) lustre-OST0001: restore index '0x1020000-OST0001' with [0x200000005:0x2:0x0]: rc = 0 [ 8909.625397] Lustre: 280814:0:(osd_scrub.c:1854:osd_index_restore()) lustre-OST0001: restore index '0x20000' with [0x200000003:0x5:0x0]: rc = 0 [ 8909.641651] Lustre: 280814:0:(osd_scrub.c:1854:osd_index_restore()) lustre-OST0001: restore index '0x2020000-OST0001' with [0x200000005:0x3:0x0]: rc = 0 [ 8909.650640] Lustre: 280814:0:(osd_scrub.c:1854:osd_index_restore()) lustre-OST0001: restore index '0x1020000' with [0x200000003:0x6:0x0]: rc = 0 [ 8909.674100] Lustre: Found index 1 for lustre-OST0001, updating log [ 8909.715342] Lustre: 280820:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 8909.813934] Lustre: lustre-OST0001: Aborting recovery [ 8913.894186] Lustre: lustre-OST0001: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x0:6 to 0x0:33) [ 8913.914970] LustreError: 280593:0:(lod_dev.c:511:lod_sub_recovery_thread()) lustre-MDT0001-osp-MDT0000: get update log duration 10, retries 0, failed: rc = -108 [ 8913.922384] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x0:6 to 0x0:33) [ 8913.946247] Lustre: lustre-OST0001-osc-MDT0000: update sequence from 0x100010000 to 0x2c0000bd0 [ 8913.991074] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000bd0 [ 8915.945641] Lustre: lustre-OST0000: new connection from lustre-MDT0001-mdtlov (cleaning up unused objects from 0x280000400:5 to 0x280000400:33) [ 8915.946860] Lustre: lustre-OST0001: new connection from lustre-MDT0001-mdtlov (cleaning up unused objects from 0x2c0000400:5 to 0x2c0000400:33) [ 8917.429891] LustreError: 280560:0:(mdt_identity.c:115:mdt_identity_do_upcall()) lustre-MDT0000: error invoking upcall /root/Work/Lustre/R5/lustre-release/lustre/tests/../utils/l_getidentity lustre-MDT0000 0: rc -2; check /proc/fs/lustre/mdt/lustre-MDT0000/identity_upcall, time 659us: rc = -2 [ 8917.929450] LustreError: 280558:0:(mdt_identity.c:115:mdt_identity_do_upcall()) lustre-MDT0000: error invoking upcall /root/Work/Lustre/R5/lustre-release/lustre/tests/../utils/l_getidentity lustre-MDT0000 0: rc -2; check /proc/fs/lustre/mdt/lustre-MDT0000/identity_upcall, time 730us: rc = -2 [ 8917.960505] LustreError: 280558:0:(mdt_identity.c:115:mdt_identity_do_upcall()) Skipped 23 previous similar messages [ 8931.313027] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 8931.324715] Lustre: Skipped 2 previous similar messages [ 8931.327157] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 8936.421755] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 8936.434162] Lustre: Skipped 6 previous similar messages [ 8937.029733] Lustre: server umount lustre-MDT0000 complete [ 8938.258597] LustreError: 280542:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1787126326 with bad export cookie 7978009849209287647 [ 8938.260119] LustreError: MGC192.168.203.122@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 8938.265455] LustreError: 280542:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 1 previous similar message [ 8941.557462] LustreError: 280558:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 8941.557478] Lustre: lustre-MDT0001-lwp-OST0001: Connection to lustre-MDT0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 8941.557970] Lustre: lustre-MDT0001: Not available for connect from 0@lo (stopping) [ 8941.600098] LustreError: 280558:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 3 previous similar messages [ 8941.606378] Lustre: Skipped 2 previous similar messages [ 8943.584878] LustreError: 280756:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 8943.586164] Lustre: lustre-MDT0001: Not available for connect from 0@lo (stopping) [ 8943.614079] Lustre: Skipped 2 previous similar messages [ 8944.872621] Lustre: server umount lustre-MDT0001 complete [ 8952.222663] Lustre: server umount lustre-OST0000 complete [ 8959.512616] Lustre: server umount lustre-OST0001 complete [ 8969.726729] Lustre: DEBUG MARKER: == conf-sanity test 109a: test lctl clear_conf fsname ==== 03:59:15 (1787126355) [ 9000.158829] Lustre: DEBUG MARKER: oleg322-server.virtnet: executing set_hostid [ 9009.829886] Lustre: DEBUG MARKER: oleg322-server.virtnet: executing load_modules_local [ 9044.701713] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 9044.879581] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 9044.963948] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 9045.061205] Lustre: lustre-MDT0000: new disk, initializing [ 9045.162345] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 9045.169383] Lustre: Skipped 1 previous similar message [ 9045.204647] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 9049.703670] Lustre: DEBUG MARKER: oleg322-server.virtnet: executing set_default_debug -1 all [ 9060.264511] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 9060.372264] Lustre: 284886:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 9060.439259] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [ 9060.445595] Lustre: Skipped 1 previous similar message [ 9060.526376] Lustre: lustre-MDT0001: new disk, initializing [ 9060.633739] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 9060.681936] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [ 9060.702808] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 9065.285414] Lustre: DEBUG MARKER: oleg322-server.virtnet: executing set_default_debug -1 all [ 9075.159328] Lustre: DEBUG MARKER: oleg322-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 9081.801617] Lustre: DEBUG MARKER: oleg322-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 9088.645433] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 9088.896212] Lustre: lustre-OST0000: new disk, initializing [ 9088.900724] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 9088.907113] Lustre: 286865:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 9088.989277] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 9095.724407] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [ 9095.735731] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 9095.766905] Lustre: DEBUG MARKER: oleg322-server.virtnet: executing set_default_debug -1 all [ 9095.866167] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 9104.595597] Lustre: DEBUG MARKER: oleg322-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 9109.381614] Lustre: Setting parameter lustre-MDT0000.mdd.atime_diff=62 in log lustre-MDT0000 [ 9118.861561] Lustre: Modifying parameter lustre-MDT0000.mdd.atime_diff=63 in log lustre-MDT0000 [ 9122.930917] Lustre: Setting parameter lustre-client.llite.max_read_ahead_mb=32 in log lustre-client [ 9128.505760] Lustre: Modifying parameter lustre-client.llite.max_read_ahead_mb=64 in log lustre-client [ 9170.914702] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 9170.919635] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 9170.939622] Lustre: Skipped 1 previous similar message [ 9170.953672] Lustre: Skipped 1 previous similar message [ 9175.547857] Lustre: server umount lustre-OST0000 complete [ 9181.164340] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 9181.169729] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 9181.180021] Lustre: Skipped 1 previous similar message [ 9181.202166] Lustre: Skipped 1 previous similar message [ 9185.726268] Lustre: server umount lustre-MDT0000 complete [ 9186.273217] LustreError: 284898:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 9186.300932] LustreError: 284898:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 2 previous similar messages [ 9189.545111] LustreError: 284879:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1787126578 with bad export cookie 7978009849209290349 [ 9189.563042] LustreError: MGC192.168.203.122@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 9189.567960] LustreError: 284879:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 9190.068463] Lustre: server umount lustre-MDT0001 complete [ 9198.403041] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 9202.931698] Lustre: DEBUG MARKER: oleg322-server.virtnet: executing set_default_debug -1 all [ 9207.289666] Lustre: server umount MGS complete [ 9219.386557] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 9219.889469] LustreError: 290632:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 9219.926634] LustreError: 290645:0:(qmt_pool.c:1639:qmt_pool_add_rem()) lustre-QMT0000: can't remove lustre-OST0000_UUID pool 'pool1': rc = -22 [ 9220.013611] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 9224.159283] Lustre: DEBUG MARKER: oleg322-server.virtnet: executing set_default_debug -1 all [ 9225.191112] LustreError: 290633:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 9233.177025] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 9238.361821] Lustre: DEBUG MARKER: oleg322-server.virtnet: executing set_default_debug -1 all [ 9246.688402] Lustre: DEBUG MARKER: oleg322-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 9252.856661] Lustre: DEBUG MARKER: oleg322-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 9260.420649] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 9260.894610] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 9260.900491] Lustre: Skipped 1 previous similar message [ 9267.679863] Lustre: DEBUG MARKER: oleg322-server.virtnet: executing set_default_debug -1 all [ 9276.485444] Lustre: DEBUG MARKER: oleg322-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 9306.084597] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 9306.095392] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 9306.127490] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 9309.155669] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 9311.723556] Lustre: server umount lustre-OST0000 complete [ 9319.396751] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 9319.424854] Lustre: Skipped 2 previous similar messages [ 9321.743597] Lustre: server umount lustre-MDT0000 complete [ 9324.514077] LustreError: 291362:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 9324.538280] LustreError: 291362:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 2 previous similar messages [ 9325.330214] LustreError: 290612:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1787126714 with bad export cookie 7978009849209292190 [ 9325.332987] LustreError: MGC192.168.203.122@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 9325.348484] LustreError: 290612:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 9325.748718] Lustre: server umount lustre-MDT0001 complete [ 9336.444897] Lustre: DEBUG MARKER: oleg322-server.virtnet: executing unload_modules_local [ 9339.390421] Key type lgssc unregistered [ 9339.697635] LNet: 294578:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 9339.717504] LNetError: 294578:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 9339.745730] LNet: Removed LNI 192.168.203.122@tcp [ 9340.659187] Key type .llcrypt unregistered [ 9340.666165] Key type ._llcrypt unregistered [ 9381.871476] Key type ._llcrypt registered [ 9381.875492] Key type .llcrypt registered [ 9381.997128] Lustre: DEBUG MARKER: oleg322-server.virtnet: executing set_hostid [ 9398.381353] Lustre: DEBUG MARKER: oleg322-server.virtnet: executing load_modules_local [ 9399.338788] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 9399.393464] alg: No test for adler32 (adler32-zlib) [ 9400.480349] Lustre: Lustre: Build Version: 2.17.57_1_g9da61ce [ 9400.838220] LNet: Added LNI 192.168.203.122@tcp [8/256/0/180] [ 9402.536342] Key type lgssc registered [ 9403.884082] Lustre: Echo OBD driver; http://www.lustre.org/ [ 9443.742441] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 9443.785183] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 9445.077976] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 9445.118336] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 9445.235568] Lustre: lustre-MDT0000: new disk, initializing [ 9445.331503] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 9445.359910] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 9450.593994] Lustre: DEBUG MARKER: oleg322-server.virtnet: executing set_default_debug -1 all [ 9461.425268] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 9461.549660] Lustre: 298539:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 9461.633299] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [ 9461.638608] Lustre: Skipped 1 previous similar message [ 9461.696164] Lustre: lustre-MDT0001: new disk, initializing [ 9461.770665] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 9461.813742] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [ 9461.822122] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 9466.551924] Lustre: DEBUG MARKER: oleg322-server.virtnet: executing set_default_debug -1 all [ 9476.124812] Lustre: DEBUG MARKER: oleg322-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 9482.148129] Lustre: DEBUG MARKER: oleg322-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 9489.565083] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 9489.861941] Lustre: lustre-OST0000: new disk, initializing [ 9489.865314] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 9489.871655] Lustre: 300505:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 9489.959732] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 9496.768640] Lustre: DEBUG MARKER: oleg322-server.virtnet: executing set_default_debug -1 all [ 9497.193739] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [ 9497.200377] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 9497.315432] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 9506.111927] Lustre: DEBUG MARKER: oleg322-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 9512.203553] Lustre: Setting parameter lustre-MDT0000.mdd.atime_diff=62 in log lustre-MDT0000 [ 9525.891282] Lustre: Modifying parameter lustre-MDT0000.mdd.atime_diff=63 in log lustre-MDT0000 [ 9529.801651] Lustre: Setting parameter lustre-client.llite.max_read_ahead_mb=32 in log lustre-client [ 9574.379161] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 9574.380990] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 9574.381632] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 9574.392533] Lustre: Skipped 1 previous similar message [ 9579.485531] Lustre: server umount lustre-OST0000 complete [ 9584.610734] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 9584.614618] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 9584.629348] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 9584.629356] Lustre: Skipped 1 previous similar message [ 9589.632792] Lustre: server umount lustre-MDT0000 complete [ 9589.729355] LustreError: 298545:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 9589.743734] LustreError: 298545:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 3 previous similar messages [ 9593.481139] LustreError: 299333:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1787126982 with bad export cookie 15537002885433348580 [ 9593.484896] LustreError: MGC192.168.203.122@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 9593.504285] LustreError: 299333:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 2 previous similar messages [ 9593.949274] Lustre: server umount lustre-MDT0001 complete [ 9603.638610] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 9608.218421] Lustre: DEBUG MARKER: oleg322-server.virtnet: executing set_default_debug -1 all [ 9613.365981] Lustre: server umount MGS complete [ 9624.926079] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 9625.429707] LustreError: 304371:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 9625.467730] LustreError: 304384:0:(qmt_pool.c:1639:qmt_pool_add_rem()) lustre-QMT0000: can't remove lustre-OST0000_UUID pool 'pool1': rc = -22 [ 9625.536643] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 9630.189345] Lustre: DEBUG MARKER: oleg322-server.virtnet: executing set_default_debug -1 all [ 9630.700333] LustreError: 304372:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 9635.827331] LustreError: 304371:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 9639.156663] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 9639.544912] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 9643.661628] Lustre: DEBUG MARKER: oleg322-server.virtnet: executing set_default_debug -1 all [ 9651.330366] Lustre: DEBUG MARKER: oleg322-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 9657.146799] Lustre: DEBUG MARKER: oleg322-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 9663.963315] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 9664.385862] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 9671.984100] Lustre: DEBUG MARKER: oleg322-server.virtnet: executing set_default_debug -1 all [ 9680.198683] Lustre: DEBUG MARKER: oleg322-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 9711.585167] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 9711.588364] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 9711.596817] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 9711.600082] Lustre: Skipped 2 previous similar messages [ 9711.612114] Lustre: Skipped 2 previous similar messages [ 9716.708849] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 9716.714923] Lustre: Skipped 1 previous similar message [ 9717.458130] Lustre: server umount lustre-OST0000 complete [ 9721.826470] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 9721.859080] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 9726.433478] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 9726.447426] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 9727.904315] Lustre: server umount lustre-MDT0000 complete [ 9731.708366] LustreError: 307097:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1787127120 with bad export cookie 15537002885433350407 [ 9731.720391] LustreError: MGC192.168.203.122@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 9731.724480] LustreError: 307097:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 9732.187769] Lustre: server umount lustre-MDT0001 complete [ 9742.767801] Lustre: DEBUG MARKER: oleg322-server.virtnet: executing unload_modules_local [ 9745.437447] Key type lgssc unregistered [ 9745.717523] LNet: 308321:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 9745.726415] LNetError: 308321:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 9745.741054] LNet: Removed LNI 192.168.203.122@tcp [ 9746.481274] Key type .llcrypt unregistered [ 9746.485312] Key type ._llcrypt unregistered [ 9766.856673] Key type ._llcrypt registered [ 9766.859539] Key type .llcrypt registered [ 9766.942379] Lustre: DEBUG MARKER: oleg322-server.virtnet: executing set_hostid [ 9781.771893] Lustre: DEBUG MARKER: oleg322-server.virtnet: executing load_modules_local [ 9783.205878] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 9783.228178] alg: No test for adler32 (adler32-zlib) [ 9784.479802] Lustre: Lustre: Build Version: 2.17.57_1_g9da61ce [ 9784.891217] LNet: Added LNI 192.168.203.122@tcp [8/256/0/180] [ 9786.600584] Key type lgssc registered [ 9787.784702] Lustre: Echo OBD driver; http://www.lustre.org/ [ 9815.418761] Lustre: DEBUG MARKER: oleg322-server.virtnet: executing load_modules_local [ 9830.079865] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 9830.135764] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 9831.417222] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 9831.483737] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 9831.633642] Lustre: lustre-MDT0000: new disk, initializing [ 9831.743478] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 9831.774368] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 9836.275810] Lustre: DEBUG MARKER: oleg322-server.virtnet: executing set_default_debug -1 all [ 9846.694782] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 9847.054728] Lustre: lustre-OST0000: new disk, initializing [ 9847.058285] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 9847.062185] Lustre: Skipped 1 previous similar message [ 9847.066371] Lustre: 312373:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 9847.135699] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 9853.252247] Lustre: DEBUG MARKER: oleg322-server.virtnet: executing set_default_debug -1 all [ 9857.097399] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:0:ost [ 9857.102246] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:0:ost] [ 9857.123660] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x240000400 [10796.368715] Lustre: server umount lustre-MDT0000 complete [10799.944728] LustreError: 311558:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1787128188 with bad export cookie 11390564223319132240 [10799.965648] LustreError: MGC192.168.203.122@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [10809.427197] Lustre: server umount lustre-OST0000 complete [10833.395886] Lustre: DEBUG MARKER: oleg322-server.virtnet: executing set_hostid [10843.484211] Lustre: DEBUG MARKER: oleg322-server.virtnet: executing load_modules_local [10880.191167] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [10880.452539] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [10880.488680] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [10880.598803] Lustre: lustre-MDT0000: new disk, initializing [10880.693871] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [10880.726895] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [10885.350951] Lustre: DEBUG MARKER: oleg322-server.virtnet: executing set_default_debug -1 all [10896.214542] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [10896.349446] Lustre: 316747:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [10896.408297] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [10896.411620] Lustre: Skipped 1 previous similar message [10896.494626] Lustre: lustre-MDT0001: new disk, initializing [10896.615959] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [10896.667460] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [10896.682481] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [10901.241521] Lustre: DEBUG MARKER: oleg322-server.virtnet: executing set_default_debug -1 all [10910.941722] Lustre: DEBUG MARKER: oleg322-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [10915.593821] Lustre: DEBUG MARKER: oleg322-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [10922.454665] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [10922.784551] Lustre: lustre-OST0000: new disk, initializing [10922.794403] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [10922.805892] Lustre: 318714:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [10922.899476] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [10929.597811] Lustre: DEBUG MARKER: oleg322-server.virtnet: executing set_default_debug -1 all [10932.377496] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [10932.392124] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [10932.576655] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [10939.038610] Lustre: DEBUG MARKER: oleg322-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [10944.249870] Lustre: DEBUG MARKER: oleg322-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 [10944.434890] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [10948.853073] Lustre: DEBUG MARKER: oleg322-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid 50 [10949.091243] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [10953.189298] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [10953.189680] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [10953.193364] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [10953.198545] LustreError: Skipped 1 previous similar message [10953.218127] Lustre: Skipped 1 previous similar message [10957.001920] Lustre: server umount lustre-OST0000 complete [10963.429622] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [10963.436915] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [10963.440516] Lustre: Skipped 1 previous similar message [10963.441247] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [10963.445236] Lustre: Skipped 2 previous similar messages [10967.253214] Lustre: server umount lustre-MDT0000 complete [10970.567331] LustreError: 316737:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1787128359 with bad export cookie 11390564223322073059 [10970.579300] LustreError: MGC192.168.203.122@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [10970.580117] LustreError: 316737:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 2 previous similar messages [10970.899626] Lustre: server umount lustre-MDT0001 complete [10974.138541] Lustre: DEBUG MARKER: SKIP: conf-sanity test_111 skipping SLOW test 111 [10975.606797] Lustre: DEBUG MARKER: == conf-sanity test 112a: mount OST with no_create option ========================================================== 04:32:42 (1787128362) [10981.305944] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [10981.637603] LustreError: 321028:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [10981.659894] LustreError: 321028:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 3 previous similar messages [10981.699950] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [10985.522038] Lustre: DEBUG MARKER: oleg322-server.virtnet: executing set_default_debug -1 all [10986.978537] LustreError: 321029:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [10992.109777] LustreError: 321028:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [10992.884267] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [10993.161749] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [10997.311321] Lustre: DEBUG MARKER: oleg322-server.virtnet: executing set_default_debug -1 all [11006.509147] Lustre: DEBUG MARKER: oleg322-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [11012.761759] Lustre: DEBUG MARKER: oleg322-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [11019.822147] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [11020.228034] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [11026.469584] Lustre: DEBUG MARKER: oleg322-server.virtnet: executing set_default_debug -1 all [11033.209446] Lustre: DEBUG MARKER: oleg322-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [11039.762051] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [11039.954578] Lustre: lustre-OST0001: new disk, initializing [11039.958453] Lustre: srv-lustre-OST0001: No data found on store. Initialize space. [11039.974066] Lustre: 323863:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [11040.078849] Lustre: lustre-OST0001: Imperative Recovery not enabled, recovery window 60-180 [11046.356903] Lustre: DEBUG MARKER: oleg322-server.virtnet: executing set_default_debug -1 all [11048.524746] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x00000002c0000400-0x0000000300000400]:1:ost [11048.540630] Lustre: cli-lustre-OST0001-super: Allocated super-sequence [0x00000002c0000400-0x0000000300000400]:1:ost] [11048.574808] LustreError: 324324:0:(osp_precreate.c:974:osp_precreate_cleanup_orphans()) lustre-OST0001-osc-MDT0001: cannot cleanup orphans: rc = -1 [11048.593276] LustreError: 324324:0:(osp_precreate.c:974:osp_precreate_cleanup_orphans()) Skipped 1 previous similar message [11049.642224] LustreError: 324328:0:(osp_precreate.c:974:osp_precreate_cleanup_orphans()) lustre-OST0001-osc-MDT0000: cannot cleanup orphans: rc = -1 [11049.650195] LustreError: 324328:0:(osp_precreate.c:974:osp_precreate_cleanup_orphans()) Skipped 1 previous similar message [11050.664830] LustreError: 324324:0:(osp_precreate.c:974:osp_precreate_cleanup_orphans()) lustre-OST0001-osc-MDT0001: cannot cleanup orphans: rc = -1 [11052.781608] LustreError: 324328:0:(osp_precreate.c:974:osp_precreate_cleanup_orphans()) lustre-OST0001-osc-MDT0000: cannot cleanup orphans: rc = -1 [11052.796150] LustreError: 324328:0:(osp_precreate.c:974:osp_precreate_cleanup_orphans()) Skipped 4 previous similar messages [11055.829607] Lustre: DEBUG MARKER: oleg322-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 [11056.070722] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [11056.933803] LustreError: 324324:0:(osp_precreate.c:974:osp_precreate_cleanup_orphans()) lustre-OST0001-osc-MDT0001: cannot cleanup orphans: rc = -1 [11056.945452] LustreError: 324324:0:(osp_precreate.c:974:osp_precreate_cleanup_orphans()) Skipped 7 previous similar messages [11061.081927] Lustre: DEBUG MARKER: oleg322-client.virtnet: executing wait_import_state (FULL|IDLE) osc.lustre-OST0000-osc-ffff93358723a000.ost_server_uuid 50 [11062.549483] Lustre: DEBUG MARKER: osc.lustre-OST0000-osc-ffff93358723a000.ost_server_uuid in FULL state after 0 sec [11065.126530] LustreError: 324328:0:(osp_precreate.c:974:osp_precreate_cleanup_orphans()) lustre-OST0001-osc-MDT0000: cannot cleanup orphans: rc = -1 [11065.139215] LustreError: 324328:0:(osp_precreate.c:974:osp_precreate_cleanup_orphans()) Skipped 14 previous similar messages [11067.506904] Lustre: DEBUG MARKER: oleg322-client.virtnet: executing wait_import_state (FULL|IDLE) osc.lustre-OST0001-osc-ffff93358723a000.ost_server_uuid 50 [11069.055275] Lustre: DEBUG MARKER: osc.lustre-OST0001-osc-ffff93358723a000.ost_server_uuid in FULL state after 0 sec [11070.401798] Lustre: lustre-OST0001-osc-MDT0000: update sequence from 0x100010000 to 0x2c0000401 [11084.773836] Lustre: lustre-OST0001-osc-MDT0001: Connection to lustre-OST0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [11084.775288] Lustre: lustre-OST0001: Not available for connect from 0@lo (stopping) [11084.786288] Lustre: Skipped 1 previous similar message [11084.794334] Lustre: Skipped 1 previous similar message [11087.171317] Lustre: lustre-OST0001: Not available for connect from 192.168.203.22@tcp (stopping) [11089.309413] Lustre: server umount lustre-OST0001 complete [11089.905706] LustreError: 324329:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [11089.919564] LustreError: 324329:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 1 previous similar message [11095.010751] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [11095.015691] LustreError: 324329:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [11095.017222] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [11095.030505] Lustre: Skipped 1 previous similar message [11095.052909] LustreError: 324329:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 2 previous similar messages [11100.542108] Lustre: server umount lustre-OST0000 complete [11105.257120] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [11105.274607] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [11105.274616] Lustre: Skipped 3 previous similar messages [11105.290626] Lustre: Skipped 1 previous similar message [11109.830627] Lustre: server umount lustre-MDT0000 complete [11110.371988] LustreError: 321025:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [11110.402271] LustreError: 321025:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 3 previous similar messages [11113.663238] LustreError: 321008:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1787128502 with bad export cookie 11390564223322073941 [11113.665330] LustreError: MGC192.168.203.122@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [11113.675899] LustreError: 321008:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [11114.086850] Lustre: server umount lustre-MDT0001 complete [11125.494183] Lustre: DEBUG MARKER: oleg322-server.virtnet: executing unload_modules_local [11128.338792] Key type lgssc unregistered [11128.650488] LNet: 326550:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [11128.659563] LNetError: 326550:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [11128.689962] LNet: Removed LNI 192.168.203.122@tcp [11129.650123] Key type .llcrypt unregistered [11129.652714] Key type ._llcrypt unregistered [11155.776279] Key type ._llcrypt registered [11155.778245] Key type .llcrypt registered [11155.925614] Lustre: DEBUG MARKER: oleg322-server.virtnet: executing load_modules_local [11156.798364] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [11156.818901] alg: No test for adler32 (adler32-zlib) [11157.879360] Lustre: Lustre: Build Version: 2.17.57_1_g9da61ce [11158.085256] LNet: Added LNI 192.168.203.122@tcp [8/256/0/180] [11159.712157] Key type lgssc registered [11160.676787] Lustre: Echo OBD driver; http://www.lustre.org/ [11168.188599] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [11168.223392] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [11169.859935] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [11174.785284] Lustre: DEBUG MARKER: oleg322-server.virtnet: executing set_default_debug -1 all [11180.005135] LustreError: 327975:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [11180.023100] LustreError: 327975:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 2 previous similar messages [11182.373666] Lustre: DEBUG MARKER: oleg322-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [11184.037449] LustreError: 327976:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [11188.498642] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [11188.821042] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [11193.123452] Lustre: DEBUG MARKER: oleg322-server.virtnet: executing set_default_debug -1 all [11201.407105] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [11201.908024] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [11208.117358] Lustre: DEBUG MARKER: oleg322-server.virtnet: executing set_default_debug -1 all [11214.188838] LustreError: 329655:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [11215.223215] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:36 to 0x280000401:65) [11215.510859] Lustre: DEBUG MARKER: oleg322-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [11220.451057] LustreError: 330275:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [11220.487093] LustreError: 330275:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 3 previous similar messages [11222.204206] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [11222.471100] Lustre: lustre-OST0001: Imperative Recovery not enabled, recovery window 60-180 [11227.645959] Lustre: lustre-OST0001: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x2c0000401:3 to 0x2c0000401:33) [11228.252297] Lustre: DEBUG MARKER: oleg322-server.virtnet: executing set_default_debug -1 all [11234.973644] Lustre: DEBUG MARKER: oleg322-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0001-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [11241.714568] Lustre: DEBUG MARKER: oleg322-server.virtnet: executing wait_import_state (FULL|IDLE) os[cp].lustre-OST0001-osc-MDT0001.ost_server_uuid 50 [11241.893648] Lustre: DEBUG MARKER: os[cp].lustre-OST0001-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [11350.514842] LustreError: lustre-OST0001-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [11350.531849] Lustre: lustre-OST0001-osc-MDT0000: Connection to lustre-OST0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [11350.562473] Lustre: lustre-OST0001: Not available for connect from 0@lo (stopping) [11355.630670] Lustre: lustre-OST0001: Not available for connect from 0@lo (stopping) [11355.639124] Lustre: Skipped 2 previous similar messages [11360.225114] Lustre: lustre-OST0001 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [11360.461305] Lustre: server umount lustre-OST0001 complete [11360.739046] LustreError: 329657:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [11360.789141] LustreError: 329657:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 1 previous similar message [11367.907764] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [11367.923277] LustreError: Skipped 1 previous similar message [11367.937293] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [11367.954018] Lustre: Skipped 1 previous similar message [11367.969844] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [11368.931260] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [11368.943319] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [11370.989669] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [11370.995138] LustreError: 329656:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [11371.009262] Lustre: Skipped 2 previous similar messages [11371.050889] LustreError: 329656:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 3 previous similar messages [11376.103417] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [11376.118026] Lustre: Skipped 1 previous similar message [11382.240172] Lustre: lustre-OST0000 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [11382.381692] Lustre: server umount lustre-OST0000 complete [11388.385899] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [11388.398069] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [11388.437445] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [11388.449777] Lustre: Skipped 2 previous similar messages [11392.611763] Lustre: server umount lustre-MDT0000 complete [11396.349832] LustreError: 331659:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1787128785 with bad export cookie 7470046545225004882 [11396.363839] LustreError: MGC192.168.203.122@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [11396.370831] LustreError: 331659:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 3 previous similar messages [11396.713337] Lustre: server umount lustre-MDT0001 complete [11408.183379] Lustre: DEBUG MARKER: oleg322-server.virtnet: executing unload_modules_local [11410.582164] Key type lgssc unregistered [11410.856105] LNet: 332921:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [11410.866417] LNetError: 332921:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [11410.888755] LNet: Removed LNI 192.168.203.122@tcp [11411.765965] Key type .llcrypt unregistered [11411.768858] Key type ._llcrypt unregistered [11463.901761] Key type ._llcrypt registered [11463.903596] Key type .llcrypt registered [11464.034651] Lustre: DEBUG MARKER: oleg322-server.virtnet: executing load_modules_local [11465.227123] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [11465.285396] alg: No test for adler32 (adler32-zlib) [11466.342348] Lustre: Lustre: Build Version: 2.17.57_1_g9da61ce [11466.577545] LNet: Added LNI 192.168.203.122@tcp [8/256/0/180] [11468.248160] Key type lgssc registered [11469.265477] Lustre: Echo OBD driver; http://www.lustre.org/ [11477.825753] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [11477.848694] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [11479.518185] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [11484.612990] Lustre: DEBUG MARKER: oleg322-server.virtnet: executing set_default_debug -1 all [11489.767367] LustreError: 335215:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [11489.808158] LustreError: 335215:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 2 previous similar messages [11493.256357] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [11493.698968] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [11497.847474] Lustre: DEBUG MARKER: oleg322-server.virtnet: executing set_default_debug -1 all [11500.497783] Lustre: Modifying parameter general.debug_raw_pointers=Y in log params [11507.332432] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [11507.663381] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [11514.298962] Lustre: DEBUG MARKER: oleg322-server.virtnet: executing set_default_debug -1 all [11518.963144] LustreError: 336705:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [11522.981614] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [11523.247772] Lustre: lustre-OST0001: Imperative Recovery not enabled, recovery window 60-180 [11525.307244] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:36 to 0x280000401:97) [11525.318665] Lustre: lustre-OST0001: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x2c0000401:3 to 0x2c0000401:65) [11529.381085] Lustre: DEBUG MARKER: oleg322-server.virtnet: executing set_default_debug -1 all [11537.759184] Lustre: DEBUG MARKER: Using TIMEOUT=20 [11544.111575] Lustre: Setting parameter general.lod.*.mdt_hash=crush in log params [11556.323046] Lustre: lustre-MDT0000-lwp-OST0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [11556.341706] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [11556.342785] Lustre: Skipped 3 previous similar messages [11556.357423] Lustre: Skipped 3 previous similar messages [11561.455518] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [11561.460341] Lustre: Skipped 3 previous similar messages [11562.602916] Lustre: server umount lustre-MDT0000 complete [11566.458615] LustreError: 335195:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1787128955 with bad export cookie 3507384682634959183 [11566.461345] LustreError: MGC192.168.203.122@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [11566.469314] LustreError: 335195:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 2 previous similar messages [11566.563957] Lustre: lustre-MDT0001-lwp-OST0000: Connection to lustre-MDT0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [11566.565900] LustreError: 335211:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [11566.570778] Lustre: lustre-MDT0001: Not available for connect from 0@lo (stopping) [11566.592324] Lustre: Skipped 1 previous similar message [11566.632238] LustreError: 335211:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 1 previous similar message [11566.895860] Lustre: server umount lustre-MDT0001 complete [11573.098767] Lustre: server umount lustre-OST0000 complete [11577.287576] Lustre: server umount lustre-OST0001 complete [11595.431983] Lustre: DEBUG MARKER: oleg322-server.virtnet: executing load_modules_local [11606.149454] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [11606.611619] LustreError: 340479:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [11606.714506] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [11611.368885] Lustre: DEBUG MARKER: oleg322-server.virtnet: executing set_default_debug -1 all [11612.142605] LustreError: 340480:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [11619.928385] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [11620.393315] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [11625.957398] Lustre: DEBUG MARKER: oleg322-server.virtnet: executing set_default_debug -1 all [11628.868735] Lustre: 341619:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify general/debug_raw_pointers=Y (mode = 0) failed: rc = -17 [11636.636035] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [11636.989331] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [11643.592557] Lustre: DEBUG MARKER: oleg322-server.virtnet: executing set_default_debug -1 all [11646.265646] LustreError: 341975:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [11646.290940] LustreError: 341975:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 1 previous similar message [11652.397751] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:36 to 0x280000401:129) [11652.501573] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [11658.222453] Lustre: lustre-OST0001: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x2c0000401:3 to 0x2c0000401:97) [11659.815696] Lustre: DEBUG MARKER: oleg322-server.virtnet: executing set_default_debug -1 all [11668.604537] Lustre: DEBUG MARKER: Using TIMEOUT=20 [11672.511274] Lustre: 343490:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify general/lod.*.mdt_hash=crush (mode = 0) failed: rc = -17 [11678.710100] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [11678.717454] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [11678.728711] Lustre: Skipped 3 previous similar messages [11678.754684] Lustre: Skipped 4 previous similar messages [11683.093327] Lustre: server umount lustre-MDT0000 complete [11683.817923] LustreError: 341211:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [11683.834164] LustreError: 341211:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 7 previous similar messages [11686.815329] LustreError: 340459:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1787129075 with bad export cookie 3507384682634960877 [11686.819762] LustreError: MGC192.168.203.122@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [11686.829961] LustreError: 340459:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [11687.148763] Lustre: server umount lustre-MDT0001 complete [11701.911845] Lustre: server umount lustre-OST0000 complete [11704.288115] Lustre: 334477:0:(client.c:2490:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1787129077/real 1787129077] req@ffff8beac2ed9500 x1873940426371968/t0(0) o400->lustre-MDT0001-lwp-OST0001@0@lo:12/10 lens 224/224 e 0 to 1 dl 1787129093 ref 1 fl Rpc:XNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 [11704.325594] Lustre: lustre-MDT0001-lwp-OST0001: Connection to lustre-MDT0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [11705.827319] Lustre: server umount lustre-OST0001 complete [11709.789427] Lustre: DEBUG MARKER: SKIP: conf-sanity test_114 skipping SLOW test 114 [11711.608842] Lustre: DEBUG MARKER: SKIP: conf-sanity test_115 skipping excluded test 115 [11713.679848] Lustre: DEBUG MARKER: == conf-sanity test 116: big size MDT support ============ 04:44:59 (1787129099) [11738.045835] Lustre: DEBUG MARKER: oleg322-server.virtnet: executing load_modules_local [11745.101631] loop0: detected capacity change from 0 to 1099511627776 [11745.475353] SGI XFS with ACLs, security attributes, quota, no debug enabled [11745.485569] *** VALIDATE xfs *** [11745.507520] XFS (loop0): Mounting V5 Filesystem [11745.561542] XFS (loop0): Ending clean mount [11746.865265] loop1: detected capacity change from 0 to 18691697672192 [11851.480663] Lustre: DEBUG MARKER: oleg322-server.virtnet: executing set_hostid [11862.632453] Lustre: DEBUG MARKER: oleg322-server.virtnet: executing load_modules_local [11901.876840] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [11902.163875] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [11902.189540] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [11902.263520] Lustre: lustre-MDT0000: new disk, initializing [11902.352821] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [11902.359387] Lustre: Skipped 1 previous similar message [11902.375829] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [11908.311849] Lustre: DEBUG MARKER: oleg322-server.virtnet: executing set_default_debug -1 all [11920.625903] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [11920.820424] Lustre: 348622:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [11920.927935] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [11920.932389] Lustre: Skipped 1 previous similar message [11921.018416] Lustre: lustre-MDT0001: new disk, initializing [11921.104628] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [11921.111761] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [11926.840598] Lustre: DEBUG MARKER: oleg322-server.virtnet: executing set_default_debug -1 all [11936.847258] Lustre: DEBUG MARKER: oleg322-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [11943.286792] Lustre: DEBUG MARKER: oleg322-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [11950.629191] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [11951.025493] Lustre: lustre-OST0000: new disk, initializing [11951.033646] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [11951.044527] Lustre: 350592:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [11952.597211] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [11952.619903] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [11952.721436] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [11957.556183] Lustre: DEBUG MARKER: oleg322-server.virtnet: executing set_default_debug -1 all [11968.159250] Lustre: DEBUG MARKER: oleg322-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [11973.601448] Lustre: DEBUG MARKER: oleg322-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 [11973.792680] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [11978.918656] Lustre: DEBUG MARKER: oleg322-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid 50 [11979.126524] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [11983.333455] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [11983.334367] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [11983.372001] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [11983.379240] Lustre: Skipped 1 previous similar message [11987.423646] Lustre: server umount lustre-OST0000 complete [11993.058745] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [11993.073512] LustreError: Skipped 1 previous similar message [11993.083750] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [11993.101036] Lustre: Skipped 1 previous similar message [11993.105821] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [12005.344319] Lustre: lustre-MDT0000 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [12005.500502] Lustre: server umount lustre-MDT0000 complete [12008.932253] LustreError: 349481:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [12008.954187] LustreError: 349481:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 1 previous similar message [12009.128724] LustreError: 349427:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1787129397 with bad export cookie 3507384682634962438 [12009.130914] LustreError: MGC192.168.203.122@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [12009.139507] LustreError: 349427:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 2 previous similar messages [12009.453805] Lustre: server umount lustre-MDT0001 complete [12018.033500] XFS (loop0): Unmounting Filesystem [12023.121467] Lustre: DEBUG MARKER: == conf-sanity test 117: lctl get_param return errors properly ========================================================== 04:50:09 (1787129409) [12030.574653] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [12031.220490] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [12031.231171] Lustre: Skipped 2 previous similar messages [12035.048792] Lustre: DEBUG MARKER: oleg322-server.virtnet: executing set_default_debug -1 all [12043.100911] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [12048.885204] Lustre: DEBUG MARKER: oleg322-server.virtnet: executing set_default_debug -1 all [12058.799479] Lustre: DEBUG MARKER: oleg322-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [12066.436615] Lustre: DEBUG MARKER: oleg322-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [12074.592883] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [12082.520917] Lustre: DEBUG MARKER: oleg322-server.virtnet: executing set_default_debug -1 all [12091.343901] Lustre: DEBUG MARKER: oleg322-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [12102.117180] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [12102.127317] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [12102.143801] Lustre: Skipped 1 previous similar message [12102.151184] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [12102.154216] Lustre: Skipped 5 previous similar messages [12104.173838] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [12107.609088] Lustre: server umount lustre-OST0000 complete [12117.770438] Lustre: server umount lustre-MDT0000 complete [12119.532445] LustreError: 354605:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [12119.578615] LustreError: 354605:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 4 previous similar messages [12122.132239] LustreError: 353131:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1787129510 with bad export cookie 3507384682634963320 [12122.136209] LustreError: MGC192.168.203.122@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [12122.144781] LustreError: 353131:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 2 previous similar messages [12122.559400] Lustre: server umount lustre-MDT0001 complete [12134.478791] Lustre: DEBUG MARKER: oleg322-server.virtnet: executing unload_modules_local [12137.325884] Key type lgssc unregistered [12137.589931] LNet: 356784:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [12137.598385] LNetError: 356784:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [12137.617493] LNet: Removed LNI 192.168.203.122@tcp [12138.396564] Key type .llcrypt unregistered [12138.401242] Key type ._llcrypt unregistered [12171.744406] Key type ._llcrypt registered [12171.748169] Key type .llcrypt registered [12171.854460] Lustre: DEBUG MARKER: oleg322-server.virtnet: executing load_modules_local [12172.629324] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [12172.707899] alg: No test for adler32 (adler32-zlib) [12173.853485] Lustre: Lustre: Build Version: 2.17.57_1_g9da61ce [12174.005774] LNet: Added LNI 192.168.203.122@tcp [8/256/0/180] [12175.624308] Key type lgssc registered [12176.853489] Lustre: Echo OBD driver; http://www.lustre.org/ [12184.642743] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [12184.658150] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [12186.115227] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [12191.249796] Lustre: DEBUG MARKER: oleg322-server.virtnet: executing set_default_debug -1 all [12195.322505] LustreError: 358353:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [12195.337023] LustreError: 358353:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 2 previous similar messages [12200.426092] LustreError: 358354:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [12200.579762] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [12200.952559] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [12205.663219] Lustre: DEBUG MARKER: oleg322-server.virtnet: executing set_default_debug -1 all [12213.853648] Lustre: DEBUG MARKER: oleg322-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [12220.240864] Lustre: DEBUG MARKER: oleg322-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [12227.048622] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [12227.391799] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [12234.051349] Lustre: DEBUG MARKER: oleg322-server.virtnet: executing set_default_debug -1 all [12243.014562] Lustre: DEBUG MARKER: oleg322-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [12255.078841] Lustre: lustre-MDT0001: Not available for connect from 192.168.203.22@tcp (stopping) [12256.228056] Lustre: lustre-MDT0001-lwp-OST0000: Connection to lustre-MDT0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [12256.232610] Lustre: lustre-MDT0001: Not available for connect from 0@lo (stopping) [12256.242586] Lustre: Skipped 1 previous similar message [12256.264110] Lustre: Skipped 1 previous similar message [12259.942744] Lustre: server umount lustre-MDT0001 complete [12260.158526] LustreError: 358349:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0001: not available for connect from 192.168.203.22@tcp (no target). If you are running an HA pair check that the target is mounted on the other server. [12265.276191] LustreError: 358348:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0001: not available for connect from 192.168.203.22@tcp (no target). If you are running an HA pair check that the target is mounted on the other server. [12265.305080] LustreError: 358348:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 2 previous similar messages [12270.390247] LustreError: 358349:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0001: not available for connect from 192.168.203.22@tcp (no target). If you are running an HA pair check that the target is mounted on the other server. [12270.421061] LustreError: 358349:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 2 previous similar messages [12270.423701] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [12270.608635] Lustre: MGS: Regenerating lustre-MDT0001 log by user request: rc = 0 [12270.615965] Lustre: Found index 1 for lustre-MDT0001, updating log [12270.669452] Lustre: 358343:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [12271.027957] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [12276.227126] Lustre: lustre-MDT0001-lwp-OST0000: Connection restored to 0@lo (at 0@lo) [12276.919633] Lustre: DEBUG MARKER: oleg322-server.virtnet: executing set_default_debug -1 all [12280.323052] LustreError: 361934:0:(obd_mount.c:234:lustre_start_simple()) lustre-MDT0001-lwp-OST0000 attach error -17 [12280.335836] LustreError: 361934:0:(tgt_mount.c:675:lustre_lwp_setup()) lustre-MDT0001-lwp-OST0000: setup up failed: rc -17 [12280.345559] LustreError: 358346:0:(mgc_request.c:614:do_requeue()) failed processing log: -17 [12280.466728] LustreError: 361945:0:(obd_config.c:2103:class_config_llog_handler()) MGC192.168.203.122@tcp: cfg command failed: rc = -17 [12280.479433] Lustre: cmd=cf001 0:lustre-MDT0001-osp-MDT0000 1:osp 2:lustre-MDT0000-mdtlov_UUID [12280.479433] [12290.917223] Lustre: lustre-MDT0001: Not available for connect from 192.168.203.22@tcp (stopping) [12291.560923] Lustre: lustre-MDT0001-osp-MDT0000: Connection to lustre-MDT0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [12291.566865] LustreError: lustre-MDT0001-osp-MDT0000: operation mds_statfs to node 0@lo failed: rc = -107 [12295.703848] Lustre: server umount lustre-MDT0001 complete [12295.996235] LustreError: 358349:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0001: not available for connect from 192.168.203.22@tcp (no target). If you are running an HA pair check that the target is mounted on the other server. [12307.059850] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [12307.158687] Lustre: MGS: Regenerating lustre-MDT0001 log by user request: rc = 0 [12307.164640] Lustre: Found index 1 for lustre-MDT0001, updating log [12307.212614] Lustre: 358343:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [12307.616878] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [12312.818119] Lustre: DEBUG MARKER: oleg322-server.virtnet: executing set_default_debug -1 all [12313.072333] Lustre: lustre-MDT0001-lwp-OST0000: Connection restored to 0@lo (at 0@lo) [12313.582448] LustreError: 362904:0:(obd_mount.c:234:lustre_start_simple()) lustre-MDT0001-lwp-OST0000 attach error -17 [12313.589456] LustreError: 362904:0:(tgt_mount.c:675:lustre_lwp_setup()) lustre-MDT0001-lwp-OST0000: setup up failed: rc -17 [12313.596609] LustreError: 358346:0:(mgc_request.c:614:do_requeue()) failed processing log: -17 [12313.602216] LustreError: 358346:0:(mgc_request.c:614:do_requeue()) Skipped 1 previous similar message [12313.641789] LustreError: 362912:0:(obd_config.c:761:class_setup()) osp: device 8 already setup: rc = -17 [12313.647518] LustreError: 362912:0:(obd_config.c:866:class_detach()) OBD device 8 still set up [12313.653539] LustreError: 362912:0:(obd_config.c:2103:class_config_llog_handler()) MGC192.168.203.122@tcp: cfg command failed: rc = -17 [12313.664541] Lustre: cmd=cf003 0:lustre-MDT0001-osp-MDT0000 1:lustre-MDT0001_UUID 2:192.168.203.122@tcp [12313.664541] [12618.582844] Lustre: lustre-MDT0001: Not available for connect from 192.168.203.22@tcp (stopping) [12618.594479] Lustre: Skipped 2 previous similar messages [12620.258792] LustreError: lustre-MDT0001-osp-MDT0000: operation mds_statfs to node 0@lo failed: rc = -107 [12620.268276] Lustre: lustre-MDT0001-osp-MDT0000: Connection to lustre-MDT0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [12620.287627] Lustre: Skipped 2 previous similar messages [12623.439977] Lustre: server umount lustre-MDT0001 complete [12623.685836] LustreError: 359356:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0001: not available for connect from 192.168.203.22@tcp (no target). If you are running an HA pair check that the target is mounted on the other server. [12623.734955] LustreError: 359356:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 8 previous similar messages [12636.731232] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [12636.857955] Lustre: MGS: Regenerating lustre-MDT0001 log by user request: rc = 0 [12636.866423] Lustre: Found index 1 for lustre-MDT0001, updating log [12636.904480] Lustre: 358343:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [12637.157613] Lustre: lustre-MDT0001: Imperative Recovery enabled, recovery window shrunk from 60-180 down to 60-180 [12641.689885] Lustre: DEBUG MARKER: oleg322-server.virtnet: executing set_default_debug -1 all [12642.281446] Lustre: lustre-MDT0001-lwp-OST0000: Connection restored to 0@lo (at 0@lo) [12645.369536] LustreError: 371553:0:(obd_mount.c:234:lustre_start_simple()) lustre-MDT0001-lwp-OST0000 attach error -17 [12645.375902] LustreError: 371553:0:(tgt_mount.c:675:lustre_lwp_setup()) lustre-MDT0001-lwp-OST0000: setup up failed: rc -17 [12645.391502] LustreError: 358346:0:(mgc_request.c:614:do_requeue()) failed processing log: -17 [12645.400212] LustreError: 358346:0:(mgc_request.c:614:do_requeue()) Skipped 1 previous similar message [12645.520948] LustreError: 371556:0:(lod_lov.c:210:lod_add_device()) lustre-MDT0001-osp-MDT0000: cannot connect to next dev lustre-MDT0001_UUID (-114) [12645.535694] LustreError: 371556:0:(obd_config.c:2103:class_config_llog_handler()) MGC192.168.203.122@tcp: cfg command failed: rc = -114 [12645.544873] Lustre: cmd=cf014 0:lustre-MDT0000-mdtlov 1:lustre-MDT0001_UUID 2:1 3:1 [12645.544873] [12964.840519] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [12964.840595] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [12964.845310] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [12964.845318] Lustre: Skipped 2 previous similar messages [12964.859338] Lustre: Skipped 2 previous similar messages [12968.303981] Lustre: server umount lustre-MDT0000 complete [12969.963536] LustreError: 358349:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [12969.989784] LustreError: 358349:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 10 previous similar messages [12971.516542] LustreError: 361539:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1787130360 with bad export cookie 1724839376722905024 [12971.531222] LustreError: MGC192.168.203.122@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [12971.534832] LustreError: 361539:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [12975.073813] Lustre: lustre-MDT0001-lwp-OST0000: Connection to lustre-MDT0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [12975.082338] Lustre: lustre-MDT0001: Not available for connect from 0@lo (stopping) [12975.086611] Lustre: Skipped 2 previous similar messages [12978.032170] Lustre: server umount lustre-MDT0001 complete [12982.763967] Lustre: server umount lustre-OST0000 complete [12994.895314] Lustre: DEBUG MARKER: oleg322-server.virtnet: executing set_hostid [13004.633673] Lustre: DEBUG MARKER: oleg322-server.virtnet: executing load_modules_local [13041.748828] Lustre: DEBUG MARKER: == conf-sanity test 120: cross-target rename should not create bad symlinks ========================================================== 05:07:08 (1787130428) [13048.312421] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [13048.673618] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [13048.698717] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [13048.766445] Lustre: lustre-MDT0000: new disk, initializing [13048.845944] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [13048.881460] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [13054.288805] Lustre: DEBUG MARKER: oleg322-server.virtnet: executing set_default_debug -1 all [13065.480866] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [13065.643855] Lustre: 382497:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [13065.729313] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [13065.739751] Lustre: Skipped 1 previous similar message [13065.816428] Lustre: lustre-MDT0001: new disk, initializing [13065.908312] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [13065.950766] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [13065.967548] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [13070.377358] Lustre: DEBUG MARKER: oleg322-server.virtnet: executing set_default_debug -1 all [13079.837552] Lustre: DEBUG MARKER: oleg322-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [13085.224827] Lustre: DEBUG MARKER: oleg322-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [13091.832665] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [13092.099712] Lustre: lustre-OST0000: new disk, initializing [13092.106111] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [13092.118143] Lustre: 384463:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [13092.232212] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [13098.617900] Lustre: DEBUG MARKER: oleg322-server.virtnet: executing set_default_debug -1 all [13099.610562] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [13099.623071] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [13099.729657] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [13108.033062] Lustre: DEBUG MARKER: oleg322-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [13116.905946] LustreError: lustre-MDT0000-osp-MDT0001: operation out_update to node 0@lo failed: rc = -107 [13116.917285] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [13116.948439] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [13116.956786] Lustre: Skipped 1 previous similar message [13117.957484] Lustre: server umount lustre-MDT0000 complete [13120.485027] LustreError: 382507:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [13120.506691] LustreError: 382507:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 4 previous similar messages [13121.344244] LustreError: 383291:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1787130510 with bad export cookie 1724839376722906977 [13121.347195] LustreError: MGC192.168.203.122@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [13121.372385] LustreError: 383291:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [13121.618845] Lustre: server umount lustre-MDT0001 complete [13135.849992] Lustre: server umount lustre-OST0000 complete [13167.625920] Lustre: DEBUG MARKER: oleg322-server.virtnet: executing set_hostid [13177.764292] Lustre: DEBUG MARKER: oleg322-server.virtnet: executing load_modules_local [13211.256530] Lustre: DEBUG MARKER: == conf-sanity test complete, duration 12934 sec ========= 05:09:57 (1787130597) [13213.212578] Lustre: DEBUG MARKER: === conf-sanity: start cleanup 05:09:59 (1787130599) === [13217.244813] Lustre: DEBUG MARKER: === conf-sanity: finish cleanup 05:10:03 (1787130603) === [13261.032434] Lustre: DEBUG MARKER: oleg322-server.virtnet: executing unload_modules_local [13264.455898] Key type lgssc unregistered [13265.022711] LNet: 390835:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [13265.033406] LNetError: 390835:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [13265.073051] LNet: Removed LNI 192.168.203.122@tcp [13266.339749] Key type .llcrypt unregistered [13266.343050] Key type ._llcrypt unregistered